Internal Filesystem: Difference between revisions

From Hackspire
Jump to navigation Jump to search
Line 84: Line 84:
===Retrieving a page===
===Retrieving a page===


There are no region headers since all the information required are stored in each unit header. To get how many blocks are there in a region, you take lnuPerRegion and multiply it with unitDataBlocks. It is worth noting that the block size is the same as the page size.
The best algorithm for retrieving page data from a FlashFX partition is to first create a lookup table resolving logical page numbers to physical page numbers and use that to retrieve the page.


For each unit, work out the range of blocks is contained in the region. Take the clientAddress and divide it by the block size to get the starting page number, add the number of blocks in the region for the end number.
To build the table, first iterate through every unit. Your table needs to hold two pieces of data: the physical page number and a sequence number. The key should be the logical page number. The table will be unitClientBlocks * lnuTotal entries long.


If the page number requested is in range, search through all the blocks contained in the unit. If not, skip to next unit header.
For each unit, read the header, then iterate through the blocks in the unit. You need to gather two pieces of information for each block in the unit: the logical page address of that block and the sequence number of the unit it belongs to. The logical page number is equal to the unit's (clientAddress / blockSize) + the block's logical address.


For each block, read the alloc information. First check if it is a free block by ANDing with 0xF000 and check if it equals 0xF000, next check if it's a valid block by checking bit 14. If it is not free and is a valid block, grab the logical block number by ANDing with 0x0FFF and adding the region starting page number. If it is equal to the requested page number, you've found the right block. Return the data found in the page.
If the page number is larger than the one already existing in the lookup table and if the sequence number is greater, the entry for that logical page number is set to the current block. Basically, the latest version of the logical page is the one with the greatest physical page number and sequence number.


Continue until all the unit headers have been searched. If not found, the page has not been allocated and you should return an empty page.
Use the lookup table to map logical page addresses to physical page addresses.
 
If a lookup table entry is empty, it means no physical page has been allocated to it and the data at that logical address is a blank page (a page filled with 0xFF s).


==Reliance==
==Reliance==

Revision as of 23:00, 24 January 2014

There are two layers involved with the internal filesystem. FlashFX Pro does wear leveling and bad block management while Reliance is the filesystem on top of it.

FlashFX Pro

FlashFX Pro maps logical addresses to physical addresses on the NAND. The physical space is divided into regions which are divided into units (erase blocks), each unit divided up into blocks (each equal to page size of the NAND). All integers are read as little endian.

The first block in a unit has a header describing the unit:

0x0 0x1 0x2 0x3
0x00 Signature
0x04 Signature
0x08 Signature
0x0C Signature
0x10 clientAddress
0x14 eraseCount
0x18 serialNumber
0x1C ulSequenceNumber
0x20 lnuTotal
0x24 lnuTag
0x28 numSpareUnits blockSize
0x2C lnuPerRegion partitionStartUnit
0x30 unitTotalBlocks unitClientBlocks
0x34 unitDataBlocks checksum

The spare area of the NAND is 1/32th of the page size and holds extra information

1 2 3 4
Alloc info ones-complement of byte 0 XOR byte 1 error-correcting Hamming code of bytes 0-2
seems to always be FF FF FF 0F for used pages, FF FF FF FF for unused
error-correcting code of second half of page data
error-correcting code of first half of page data

Checksum

The checksum in the unit header is calculated by adding all the bytes in the header mod 2^16.

uint16_t checksum(void *_ptr, size_t size) {
	uint16_t sum = 0;
	uint8_t *ptr = _ptr;

	while (size--) {
		sum += *ptr++;
	}

	return sum;
}

Allocation information

This contains the status and logical address of the block. A unit header will have a magic signature (0x48E2).

Status

Bits 12-15 indicate the status of the block.

Probably irrelevant for this version of FlashFX (always 0x4).

Logical address

Mask with 0x0FFF to get address.

Retrieving a page

The best algorithm for retrieving page data from a FlashFX partition is to first create a lookup table resolving logical page numbers to physical page numbers and use that to retrieve the page.

To build the table, first iterate through every unit. Your table needs to hold two pieces of data: the physical page number and a sequence number. The key should be the logical page number. The table will be unitClientBlocks * lnuTotal entries long.

For each unit, read the header, then iterate through the blocks in the unit. You need to gather two pieces of information for each block in the unit: the logical page address of that block and the sequence number of the unit it belongs to. The logical page number is equal to the unit's (clientAddress / blockSize) + the block's logical address.

If the page number is larger than the one already existing in the lookup table and if the sequence number is greater, the entry for that logical page number is set to the current block. Basically, the latest version of the logical page is the one with the greatest physical page number and sequence number.

Use the lookup table to map logical page addresses to physical page addresses.

If a lookup table entry is empty, it means no physical page has been allocated to it and the data at that logical address is a blank page (a page filled with 0xFF s).

Reliance

TODO

Resources

Goplat's post on Omnimaga

Patent for FlashFX