Hello,
I am making an open source privacy-first fitness band for myself and I am writing the firmware now as someone relatively inexperienced at firmware development (I am an electronics engineer by trade). I get it done but sometimes I run into concept issues, especially when I start overthinking, like now that I need help with.
I have a macronix SPI NOR flash on-board that I want to use as offline activity saving, backup at low battery, etc… I am dreaming up the data structure for it. Here is the values I need to save to not lose information and what will be required for my supported features in the Bluetooth Physical Activity Monitor Service:
struct memory_map_nor {
time_t timestamp;
uint16_t sub_sess_id;
uint32_t steps: 24;
uint8_t bpm;
float16_t spo2;
uint16_t pulse_inter_beat_interval;
uint16_t cadence;
uint16_t speed;
uint16_t activity_level;
uint16_t activity_type;
uint16_t temp;
};
So from this datastructure, it has a total of 28 bytes of data. This has to fit on a 256 byte page, which means 9 “rows” of data can be written per page, 144 per sector, 2304 per 64 bit block, and 147456 in total for a 32Mbit NOR.
But, I am getting confused while reading about memory structures in “normal” processors that need to read everything in 4/8-byte words via the parallel interfaces. This means that conventionally, everything has to be padded to neat structures that are divisible by 4 (32-bit) for QSPI reading. In that case, I would either have to add another 32 bits of data or pad 32 bits to every “row”, making a neat 8 data “rows” per page.
OR, because I am only using single lane SPI, would this not matter and I could shove an extra datapoint in each page. The difference is 147456 data rows vs 131072 data rows. At 3s polling rate, that is 5.12 days vs 4.55 days. For my application, the difference might be useless anyway, but the band goal battery life is 2 weeks or so.
Again, maybe I am overthinking this and can just pad the data to make everything neat and fit well. Anyone have any opinions? Thanks!
You should rearrange your structure to group the types from biggest to smallest, otherwise you get padding you didn’t want. BPM and steps for example.
Qspi in general can read single bytes. But we don’t know what controller you use.
Also you probably shouldn’t just read it into memory and cast it to your structure. If you ever change the layout everything breaks apart.
Thanks for taking a look!
Intuitively for me, steps + bpm should be next to each other because the compiler will use
bpmas the padding for the 24 bitsteps. I intentionally did it that way. At least when I checked the memory addresses when testing it that was the case (there was no padding added). Wouldn’t it be potentially more problematic to have a bit field with a weird bit number, 24, followed by a 16 bit member that can’t be “fit” into the 32 bits that the compiler wants to assign? or is that not how it works.I’m not quite sure what you mean by your last point. The flow would go: acquire data -> add to structure -> fill up a page worth of data (or a sector) -> write to memory. Then pulling it out would be: read from memory -> put in structure -> process -> send data via bluetooth. If I change the layout of anything, that would require a reflash of the MCU and previous data would already have been transferred over bluetooth (assuming end-user OTA flashing or just being in a vicinity of a phone and not out and about where memory saving is necessary) and would no longer be needed to be stored/pulled from memory. Or is there another case that I am totally missing?
So for padding, it sometimes depends on how your compiler works, but usually, it doesn’t pack bytes by default - that needs to manually be done. Otherwise, a uint32 followed by 2 uint16s, for example, will take up the space for 3 uint32s (in a 32-bit native compiler). If you manually specify packing (implemented differently depending on your compiler and such), then it will pack those all properly into just 2 uint32s.
I do imagine 24 bits followed by 16 more in a bit field for a 32-bit number would potentially cause problems. But it’s late here and I could certainly be wrong so take that with a grain of salt.
That also said, I typically don’t use bitfields directly in structures - it’s not usually good practice, at least where I work. I’d either do a uint8[3] or use a whole uint32 that is a union, and in the union would be your :24 followed by a reserved : 8, if that makes any sense. It’s sometimes worth it to leave a few extra bytes in there just from an organization standpoint.
The way I have done it in the past. Rearrange all struct members from the largest to smallest data size. Do a sizeof the struct to make sure it is what you expect. You can look at the memory map too if your compiler provides it. Point an unsigned char (uint8_t) pointer to the first member of the struct or the variable name of the struct casted as a uint8_t. Then move the data into whatever the EEPROM or flash pages the data needs using the sizeof the struct and the size of the page for the non volatile storage. Sometimes I make a temporary array of the structs if I’m going to do several on a page. Then either copy directly into my ram overlay or just return the data if it is like a lookup or whatever.
My recommendation is that you use a file system like littlefs which also has wear leveling built-in, and you don’t have to worry about any specifics of the chip
Hmmm, I used littlefs for SD card writing at work with an STM32F0 chip. It was hell working with files when tons of essential functions like appending and seeking simply didn’t work in the STM HAL… Plus dealing with opening and closing files and appending files and having to seek in them to find what you want, parsing results, cleaning old files, etc… compared to simple circular buffer and a start and end address of relevant data that can be erased once every day or week depending on use. Even with a daily erase of the NOR chip, they are rated for 100k program/erase cycles which would be over 250 years before degradation starts. I am not dealing with a ton of data nor the flexibility of a full UI/ app storage where I would definitely just use littlefs.
I’m going to make an assumption that this is C. The compiler should pad structures for you
