libfat : maybe you need to loose some weight :D

Written by cafealpha 3 comments
Classified in : Saturn, Programming, Mylife Tags : saturn, memcart, kai, fat32, library
So as you may know, I'm (still) developing the hardware/firmware/everything else for my SD card-based Saturn Cartridge.

One the main module for the firmware of this cartridge is File System access library.

As you may know, there are many File System existing in real worls : FAT, NTFS, exFat, ReiserFs, etc.
And in my project, I choose to use FAT32 in order to access files and directories.
Why FAT32 ? Just because it offers the best rate of advantages vs disadvantages in my project scope.
Advantages :
 - Available everywhere, from Windows 98 to every Linux distribution.
 - Detailed (?) specs available on MS website, lots of free source code available elsewhere.
 - Little resource requirement (at least 1~2KB of work RAM, and few tenths of program memory for code).
Disadvantages :
 - A little hard to understand why internal data structure is so messy ... more understandable after reading FAT16 specs.
 - Complex Long FileName support, and patented by MS too.
 - Write performances may be poor under certain conditions. Example : fragmented file system + large partition size.
 - Free source code available everywhere is sometimes buggy / bloated.

At the very beginning of the project (nearly 4 years ago !), I choose DharmaniTech FAT32 library , and it was easy to use, so that I could read files without many problems on software development side.
But it only supports <512 bytes file write, and a lot of advanced features weren't available, so I then switched to thinfat32 library .
Thinfat32 library was a pain to adapt to Saturn, and some advanced features weren't available, or didn't worked, so I rapidly switched to libfat .

Libfat is great, works without any major trouble, and even supports FAT16
So I kept libfat as FAT32 library for around 3 years.
But one problem of libfat is ... its weight (<- yeah, I wanted to put this pun somewhere ).
 - Sector caching is cool, but library doesn't work without it, so we need at least 8KB of work RAM for it.
 - Source code is very complete, but also very complex, so unless you aren't a talented developer, so can't customize it easily ... and unfortunately I'm not a talented developer
 - Library supports nearly everything for file/directory manipulation, so size after compilation is a little big (around 50KB IIRC).

You may ask : "why complaining about few KBs of RAM ? Saturn got plenty of RAM to use !"
Well, yes and no. My project (on game game backup from/to SD card) of resources :
 - Work RAM : 24KB ... not contiguous : 16KB+8KB
 - Program memory : work RAM, or cartridge ROM (1MB). I choose to run program directly from ROM.

And, in a (maybe) future hardware project, I may need to port FAT32 library to microcontroller, so few resources will be mandatory.

So last month, I dropped libfat, and started to look for a simpler & better FAT32 library.
When selecting a starting point for development, my choice pointed to thinfat32, not because it was perfect, but because I had some experience with it in the past, so that I know where I should start to fix things

First, I dropped some features I don't need :
 - Long FileName when writing files : LFN can be read, but only 8.3 files can be written ... it's enough for my needs
 - Long path : changed maximum size for file path from MAX_PATH (260 bytes) to 96 bytes ... it's enough for me, and saves some RAM.
 - Only one file open at once : it saves some RAM.
 - Other ? (don't remember)

Then started to fix/debug/etc the library :
 - Big/low endian support ... Saturn and PC endianness is different
 - Directory creation fix.
 - File creation in non-root directory fix.
 - Added test bench under windows ... there was one stub in original sources, but with very limited test range ...
 - Better log support.
 - FS info support.
 - FAT16 support ... not working yet (), but at least it works better than in original sources
 - File append fix ... original sources support file append ... only in the case source file is multiple of 512 bytes ... this bug took me many time to spot
 - Clock support when setting file timestamp ... original sources supported only fixed (dummy) timestamp.
 - File listing within a directory.
 - Source code cleanup.
 - MANY fixes.

Complaints about thinfat32 library may look severe, but I really thanks its author to publicly release it !
Thanks to him, I know have something that fits my needs, and have a deeper knowledge regarding FAT32 structure

Last but not least, here are some screenshots :
data/images/20151021_thinfat_testprog.tb.png
Test program contents :
In this screenshot, I first create one directory,
then write 3 files inside it.


data/images/20151021_thinfat_testbench.tb.png
Test program execution - just in order to show
a command line screenshot on my blog

data/images/20151021_thinfat_testresults.tb.png
Result in (virtual) test drive :
one directory and 3 files are created !

This is just the beginning of the development : I still need to test under Saturn emulator (yabause ), then on real hardware (Saturn + custom cartridge ).

Read more libfat : maybe you need to loose some weight :D

Backup Manager - Early Version #1

Written by cafealpha 8 comments
So I'm currently making a Saturn Backup Data Manager utility.
What's this ? It is a remake of "Memory Manager" utility present in Saturn BIOS.
data/images/20150204_backup_manager_sega1.png
Sega Memory Manager utility : main screen

It already exists, and can be used from BIOS, so why making such an utility ?
I have some reasons that pushed me making this, and one of theses is that Memory Manager user interface is very cheap, especially, it doesn't allows to select multiple backup data and copy/delete them. (Dear Sega, why didn't you developed this feature ??)
Also, there are some new features I would like to add, such as hex-edit of memory data ... I have very few time to play now (real life, children, cancer, etc), so 999 lives in Backup data is a must-have feature

This utility is currently WIP, so no download link is available right now, but it is WIP enough to publish screens on my blog
So here is what it looks like now :
data/images/20150204_backup_manager1.png
Backup Data Manager Main Screen

Don't trust the development progress only by looking at the main menu : everything behind is far for being finished
But at least, menu library itself is more or less completed, and I'm very glad about this point.

Here is a comparison of data selection screens :
data/images/20150204_backup_manager_sega2.png
Data Selection (Sega)
data/images/20150204_backup_manager3.png
Data Selection (this utility)

Yeah, I know that Japanese text is not displaying properly ... this will be fixed, but the priority for this is low, so please be patient.
And what is the green text display on Backup Data Manager screen ? This is multiple items selection

As a bonus, here are screens of Backup Manager running on real hardware.
Pictures were taken last September, when I still had a modchipped Saturn ready for use ... after that I was busy with medical stuff, and moving to a new apartment, so my Saturn I used for testing is currently hidden in a cardboard ...
data/images/20150204_backupmngr_hwtest1.tb.jpg
Early version of Backup Manager, picture was taken 4 months ago.
Can you spot the differences with current version ? ^^


data/images/20150204_backupmngr_hwtest2.tb.jpg
SatLink running on my 8 years old laptop, connected to my Saturn via USB dev cart.
This development environment is by faaar more convenient than $500+ vintage
development hardware/carts/whatever floating on ebay ♪



Among the things I would like to add before a release, there is the Saturn FDD support ... but I don't own Saturn FDD unit, and the cheapest unit I could find on auction sites costs 10'000 Yens, and the seller doesn't guarantee that it still works ...
I won't pay 10K Yens for a vintage objet I will only use once a year (and I fear my wife would trash it if she founds it ), so if somebody is ready to give some time for beta testing with Saturn Floppy Disk Drive, please let me know in the comments

Read more Backup Manager - Early Version #1

Rss feed of the tag