[MANUAL] SataPortMap, DiskIdxMap, sata_remap, SasIdxMap (& Co: internalportcfg, esataportcfg and usbportcfg)

[allexmail]

SataPortMap, DiskIdxMap, sata_remap, SasIdxMap (and Co: internalportcfg, esataportcfg and usbportcfg)

Вы можете загрузить эту статью на русском языке в формате PDF здесь.
You can download this article in Russian in PDF format here.

Initially, I wanted to publish this material on xpenology.com However, the existence of xpenology.com it remains questionable at the moment. That's why I'm posting it here, on my favorite bootloader's page.

Fact: The problem of hard disk numbering in DSM sometimes arises...

Used:
Bootloader: tinycore-redpill.v1.2.6.7.m-shell
DSM 7.3.2-86009

...Especially when using motherboards or controllers that have many SATA ports on board, but only a fraction of them are physically wired. And the problem becomes very acute when physically existing ports are numbered not from port 0, but, say, from the fourth. In most cases, the bootloader handles port numbering automatically, and you don't need to change these parameters. But if the problem arises, then here begin the "Jump through hoops".

Look at this picture. It shows: 1 built-in controller (00:17) and 4 PCI-e-Sata controllers (03:00…):

Note

Note that in the tinycore-redpill.v1.2.6.7.m-shell bootloader it is indicated that all non-existent ports (DUMMY) are indicated in red. However, it seems that this is not the case. In the image, black indicates non-existent ports. More precisely, they exist (supported by the chip), but are not physically wired on the board. Green indicates the ports to which HDDs are connected. All other ports (existing, but without connected HDDs) are hidden. Perhaps this is a drawback of the bootloader and I think we will discuss this with the author of the bootloader a little later. Or maybe I just misunderstood something.

Jumping ahead:
With this configuration, if you do not change the SataPortMap and DiskIdxMap parameters, DSM (at the installation stage - this is important!), saw the following HDD numbering from me: 6, 14, 15:

Loader configuration without manual edits (open to see)

=================================================================================

So, important bootloader parameters SataPortMap, DiskIdxMap, sata_remap, and SasIdxMap.

=================================================================================

SataPortMap

SataPortMap is a kernel command line (cmdline) parameter in XPEnology bootloaders (RedPill, ARPL, Arc), which rigidly defines the number of active SATA ports for each controller, visible to the DSM operating system. That is, it indicates the total number of ports that DSM must recognize and declare for each physical or virtual SATA controller. This is the "upper limit" or the maximum number of HDDs that DSM will look for on a given controller. It does not matter whether an HDD is physically connected to them or not.

Example: SataPortMap=8 tells the system: "On the first controller, DSM will reserve 8 slots (ports)".
Example: SataPortMap=8555 tells the system: "On the first controller, DSM will reserve 8 slots (ports), on the second, third, and fourth controllers, 5 slots each".

That is, each digit denotes one controller, and the sequence of these digits is the sequence of controllers in the order detected by the kernel. In this example, DSM will reserve 23 ports. If one HDD is connected to port 1 of the first controller and one HDD is connected to port 1 of the second controller, and all other ports are "empty", then DSM will see HDD 1 and HDD 9.

Note

Please note that the SataPortMap parameter indicates the number of ports, not their number/index. This is why the count starts with 1.

DiskIdxMap

DiskIdxMap is a kernel command line (cmdline) parameter in XPEnology bootloaders (RedPill, ARPL, Arc), which sets the starting sequence number (index) of HDDs for each SATA controller in the DSM interface. This allows you to skip certain indexes (ports) to force DSM to start displaying HDDs from the number you need.

Why is this needed?
The thing is that controller chips are designed by the manufacturer for a certain number of ports, but manufacturers of motherboards or external controllers, due to some of their limitations, do not wire all the connectors for this controller on the board, but only a part. Usually (but not necessarily) this is an even number 2, 4, 6, 8, etc. Thus, the rest of the ports simply exist inside the chip, but are inaccessible to the user. And everything would be fine, if not for one thing. Motherboard and other controller manufacturers do not always physically wire the connectors starting from the zero port of the controller. In some cases, they wire, for example, from the second, or fourth port. It turns out that ports 0-1 or 0-3 just hang in the air and are not physically available to the user. In the first image, this is clearly visible. We have already found out that all DUMMY ports are displayed in black, look at controller 0:17 - ports 00-03 cannot be used because there is no connector on the motherboard to connect an HDD to them. In turn, the remaining controllers have 5 ports each, but the DUMMY ports start from the third port for each controller (10-12, 15-17, 20-22, 25-27).

Note

Please note that the numbering for each controller is specified in hexadecimal format, i.e. two characters per controller (1 byte per controller).

Example: DiskIdxMap=04 - tells the system: "Start counting HDDs on the first controller from index 4, skipping 0, 1, 2, 3" - because 0, 1, 2, 3 do not physically exist - they are not wired on the motherboard.

A hypothetical example of a DiskIdxMap for 4 controllers (to make it clear how to count):
For the first controller, HDDs (physically existing ports) start from 4, for the second from 13, for the third from 0, for the fourth from 9. The start is considered 0.
Index calculation:

• 1st controller: (4 in decimal) = 04 in HEX
• 2nd controller: (13 in decimal) = 0D in HEX (since 10=A, 11=B, 12=C, 13=D)
• 3rd controller: (0 in decimal) = 00 in HEX
• 4th controller: (9 in decimal) = 09 in HEX

Final parameter:
DiskIdxMap=040D0009

Note

Please note that both in SataPortMap and DiskIdxMap the numbering (sequence) of the controllers themselves is as determined by the bootloader itself.

Note

Please note that these two parameters (SataPortMap and DiskIdxMap) are always used in pairs if they are used.

Let's go back to the first image. At first I thought that it was enough to edit the SataPortMap and DiskIdxMap parameters and everything would fall into place. Based on the logic I described above, my configuration should have been this:
SataPortMap=82222
DiskIdxMap=0400000000

What do I expect? I expect DSM to start counting ports from port 5 of controller 00:17 (i.e. HDDs 1-4), and then sequentially two ports from each of the 4 controllers. That is, I expect DSM to tell me: "I am ready for installation and you have HDDs 2, 7, 8". After booting DSM for installation, all my theory about SataPortMap and DiskIdxMap just falls apart:

Taaa-Daaam (open to see)

Yes, I didn't expect that either. What happened in my opinion: for some reason, DSM decided that controller 04:00 should be used first (since HDDs are connected to it), and put the internal controller 00:17 at the end of the queue, not forgetting to reserve two ports from the empty three external controllers. Please take my word for it, according to the logs (RS232) DSM accurately detects controller 00:17 first, just like the bootloader itself. Please note that DSM did not say that you have HDDs 3, 4, 10 (as logically should have been the case, because controller 03:00 is empty), if we assume that it put the internal controller 00:17 last. The numbering 1, 2, 10 was a complete surprise to me.

Jumping ahead:
Perhaps this happened because DSM identified my PCI-e controllers as eSATA (more on this later), but this point needs to be discussed with the bootloader developers. I fiddled for several hours manipulating the SataPortMap and DiskIdxMap numbers, switched the wires, made the numbering according to the cables so that controller 00:17 was listed last (used parameters SataPortMap=22228 DiskIdxMap=0000000004). I did not achieve any adequate sequences of HDDs. Each time the numbering result was unpredictable.

sata_remap

The sata_remap parameter is a specific kernel command line argument in modern XPenology bootloaders (mainly in Arc and RR/ARPL) that allows you to reassign (remap) physical SATA ports to specific logical positions in the DSM operating system.
sata_remap is a tool for per-port control of HDD indexes. Unlike DiskIdxMap, which shifts the numbering for the entire controller at once, sata_remap allows you to specifically indicate, for example: "The HDD from controller N1 port N2 should become HDD N1 in DSM".
The parameter is a list of value pairs separated by commas or colons (depending on the bootloader implementation), in the format:
physical_port_index > target_index_in_DSM

Why is this needed?

• Fixing chaotic numbering: As I said, often on motherboards, physical SATA connectors are not wired in order. sata_remap allows you to programmatically arrange them in DSM so that HDD 1 corresponds to the first physical slot in the case.
• Combining ports from different controllers: You can make HDDs connected to different chips (for example, Intel SATA and a third-party JMicron controller) appear in DSM as a single sequential group.
• Hiding a specific port: You can remap the port into which the bootable USB flash drive or service drive is inserted to a very high index (for example, 24), so that it does not take up space in the main storage interface.

Example 1: Swap two HDDs
Suppose HDD 1 (port 0) and HDD 5 (port 4) are detected by the system in the wrong order, and you want to swap them.
sata_remap=0>4:4>0
This means: "port 0 now becomes port 4, and port 4 becomes port 0".

Example 2: Complex remapping
sata_remap=1>5:2>6
This will swap port 1 with port 5, and port 2 with port 6. That is, DSM will see HDDs 5 and 6.

When to use sata_remap?
Unlike SataPortMap and DiskIdxMap, which are standard tools for configuring most systems, sata_remap is used only in specific, complex debugging scenarios when standard mapping methods do not help achieve the desired order of HDDs.

For the example mentioned above (SataPortMap=82222 and DiskIdxMap=0400000000), (skipping the first 4 ports and sequential numbering of HDDs for all controllers), sata_remap is the only solution:

"sata_remap": "4\\>0:5\\>1:6\\>2:7\\>3:8\\>4:9\\>5:13\\>6:14\\>7:18\\>8:19\\>9:23\\>10:24\\>11"

Here the numbering for DSM will be from the 1st HDD of the internal controller (which is on port 4 of the internal controller 00:17), port 5 - HDD two, etc.

Note

Please note that in the bootloader, ports are numbered from 0, and in DSM, HDDs start numbering from 1.

Note

Please note how the string is written in the configuration file specifically for this bootloader - two \\ slashes are used. When the bootloader is built, one slash is removed and the string becomes:

sata_remap="4\>0:5\>1:6\>2:7\>3:8\>4:9\>5:13\>6:14\>7:18\>8:19\>9:23\>10:24\>11"

Some sources write that sata_remap can be used ONLY for those ports where HDDs are physically connected. This is not true, at least in the modern bootloader tinycore-redpill.v1.2.6.7.m-shell. You should list all possible ports that will be used now or are planned to be used in the future. If you built DSM with an equal number of ports and HDDs, and then decided to add more controllers and HDDs to them, you need to add new ports to sata_remap and rebuild the bootloader.

Note

Please note that in this case, the new controllers being installed must not violate the existing numbering of controllers, otherwise the array in DSM may be destroyed. That is why you should plan and describe the maximum possible number of ports before installing DSM and not change it later. That is, install as many controllers as possible at once.

The tinycore-redpill.v1.2.6.7.m-shell bootloader has an automatic sata_remap function:

However, you need to take into account that it will insert into the configuration file only those ports to which hard drives are actually connected (compare with the first image of the article):

Automatically inserted parameters (open to see)

Note

Well, please note, in my case it worked incorrectly and started numbering ports from 23rd.

So, to save my situation (the first image of the article), I had to manually compile the sata_remap parameter as I indicated above:

"sata_remap": "4\\>0:5\\>1:6\\>2:7\\>3:8\\>4:9\\>5:13\\>6:14\\>7:18\\>8:19\\>9:23\\>10:24\\>11"

Here's what DSM saw, which is what I needed:

The result I was looking for (open to see)

internalportcfg, esataportcfg and usbportcfg

Note

Please note that at the DSM INSTALLATION stage, all specified HDDs may be perfectly visible to DSM and have the correct sequence. However, after installation and booting into DSM, you suddenly discover that the HDDs on external PCI-e controllers are not visible. This happens because DSM somehow defines them as removable eSATA drives (this is easy to check by going to the DSM settings in the "Information" section and looking for the model of the hard drive installed on the external controller there).

To force DSM to see all HDDs as internal, you need to use options in the sinoinfo{} section in the bootloader:
"internalportcfg": "0xffffffff",
"esataportcfg": "0x0",

internalportcfg - specifies the specific numbers (positions) of the ports that the system should consider internal. 0xffffffff is a mask where all 32 bits are set to "1". Each letter f in the hexadecimal system corresponds to four ones (1111) in binary. One f = 1111 (4 ports). In this case, the DSM system will consider the first 32 ports internal. This is the maximum number of ports, for example for DS3615xs, DS3617xs, DS3622xs+.

Example: "internalportcfg": "0xfffff0ff",

The mask 0xfffff0ff in binary form (for 32 ports) will look like this: 11111111 11111111 11110000 11111111.

What will happen in DSM?

• Ports 1 through 8: Internal (bits set to 1).
• Ports 9 through 12: NOT internal (bits set to 0).
• Ports 13 through 32: Internal (bits set to 1).

Such a "hole" is usually used in two cases:

• eSATA configuration: If you want ports N9, 10, 11, and 12 to work as external eSATA connectors. In this case, you must register them in the esataportcfg parameter (for example, 0x00000f00) so that DSM correctly identifies them as external ports for hot swapping.
• Skipping "non-existent" ports: Sometimes, on specific hardware or when using multiple controllers, numbering conflicts arise. By removing bits from internalportcfg, you force DSM to ignore these positions when looking for internal drives.

As you can easily guess, the esataportcfg and usbportcfg parameters adjust which ports will be eSATA and/or USB.

Caution

The bits in internalportcfg, esataportcfg, and usbportcfg must not overlap. If you set 0 in internalportcfg for ports 9–12, the system expects you to either leave them empty or assign them to one of the other two categories (esataportcfg or usbportcfg).

In our example:
"internalportcfg": "0xffffffff",
"esataportcfg": "0x0",

0x0 - a mask in which DSM will assume that there are no eSATA HDDs in the system.

For a 64-bit mask, some modern versions of bootloaders and DSM models (for example, on the SA6400 platform) support extended masks. HEX format: 0xffffffffffffffff. This means: All 64 ports are marked as internal.

You shouldn't set the mask larger than needed!
Although 0xffffffff is a "universal" maximum, it is important to remember about conflicts:

• USB ports: If you specify all bits as internal, DSM may try to initialize the USB controllers as SATA, which will lead to errors or invisibility of the bootloader flash drive.
• Model limits: If the synoinfo.conf has a maxdisks="12" limit, then a mask wider than 12 ports (i.e., above 0xfff) simply won't make sense, although it won't hurt.

Important

Since all HDDs are visible at the time of DSM INSTALLATION, you will most likely not pay attention to these two parameters and will edit the bootloader after installing DSM. In theory, after each edit and build, the bootloader should update all parameters in /etc.defaults/synoinfo.conf, however, in my case, these two parameters did not change by the bootloader, they remained the same as those installed during the DSM installation. After editing the bootloader configuration and building it, I had to manually edit /etc.defaults/synoinfo.conf in DSM via SSH and only then DSM saw all the HDDs.

SasIdxMap

SasIdxMap is a kernel command line parameter (cmdline) in XPEnology bootloaders (RedPill, Arc, RR) that sets the initial sequence number (index) for HDDs connected via SAS controllers (HBA). That is, a parameter that defines the numbering offset for SAS/HBA controllers in the DSM interface. It works similarly to DiskIdxMap, but applies only to those controllers that use SAS drivers (for example, the popular IT-mode LSI/Broadcom controllers).

Format: Hexadecimal number (HEX). Most often specified as one byte (2 characters) per controller.
Logic: The value 00 corresponds to HDD 1 in DSM.
Example: SasIdxMap=08 - this will tell the system that the first HDD on the SAS controller should be displayed under number 9 (since index 8 in HEX is the 9th slot in the decimal system).

Why is this needed?

• Separation of SATA and SAS: By default, DSM may try to assign SAS HDDs the same slot numbers (indexes) as built-in SATA HDDs. This leads to conflicts in which some HDDs are simply not displayed.
• Ordering in bays: If you have built-in ports on the motherboard and a separate SAS card, using SasIdxMap you can "move" the SAS HDDs to the end of the list (for example, after all SATA slots) to maintain logical numbering.

For correct operation of the disk subsystem, SasIdxMap is used in conjunction with SataPortMap and DiskIdxMap (for SATA controllers), as well as internalportcfg (to mark SAS ports as internal).

I personally have not yet had the opportunity to check this parameter, so all information about this parameter is information from the Internet.

🄯 allexmail aka allexnew
Please, when publishing, leave a link to the source of the article.