The quick lazy brown dog jumps over the foxes ?

Written by cafealpha no comments
The quick lazy brown dog jumps over the foxes ?
The brown fox quick lazy jump jumps over the ?
The ... well, I forgot what it was, but just to say that I spent a some time in messing with fonts in my Save Data Manager application
In fact, it would be more accurate to speak about code pages (*), because I finally added support for Japanese text display :]

(*) For those who forgot what code pages was :
Once upon a time, before Unicode unified every characters, (and also made strlen function not working correctly on UTF-8 text), there was a custom for all countries in the developed world to define up to 128 extra characters that would be used in second half of ASCII character table.
Of course, Japan wanted to do the same, so it added poor looking "1 byte kana" characters on this extra 128 character page, and also added the remaining characters (including kanjis) after that, which gave birth to Shift-JIS encoding.

Since Saturn was developed, during this prehistoric "codepage" era, it is normal to find can find some code page remnants in Saturn software

One of these "code page remnants" is the comment string stored in each Saturn saves : didn't you noticed that sometimes BIOS displays Japanese characters in its save management screen ?

So, Save Data Manager must display Japanese characters instead of garbage characters when showing comment in save data selection menu !

data/images/20150704_save_delete_menu.png
What save comment looked like before
adding Japanese characters display ...
(It's a 6 months old screenshot)

How to do this ?
First, I need to verify that BIOS was defining characters according to Shift-JIS charset, and the best way to do this was to modify some save states and see what BIOS is displaying on screen.

And if possible, displaying as much as possible characters at once would improve the job's efficiency, so I choose to modify Keio Flying Squadron saves since they take only 32 bytes each.

So here comes the reason I choose a such weird title for this article : I wanted to display all font characters at once on screen
In the facts, I didn't used such a complex "lazy brown dog jump blah blah blah" sentence. I just displayed all the character in ascending order, and here is what it looked like in BIOS screen :

data/images/20160203_chars_katakana_raw.png
Repeat after me : -アイウエオカキクケコサシスセソ...
Looks good  That's definitely 1 byte kana characters in typical order.

Saturn BIOS also has ability to display European (accented) character, and switching from a codepage to another is just the matter of changing a byte per save data, so let's try and see what it looks likes :
data/images/20160203_chars_francais_raw.png
Same text as above, I just changed the language value

Oh, it looks like the good old ISO 8859-1 (latin1) charset

The picture above was displayed by setting French (my native language) language code in save, and setting to another language code (I tried with Spanish) gave unsurprisingly the same results.

So in the Saturn world, one byte equals one character, and there are two codepages to handle in order to display save comment string.
I first feared I had to deal with multi-byte characters (that is a real pain to handle when coding software), but that's not the case, and in fact it's relatively easy to do.

(2 hours later)
data/images/20160203_font_katakana.png data/images/20160203_font_francais.png
Katakana and European fonts

Hmm, theses characters are looking cute :)
It's seems that it is time to feed my image conversion tool ^^

(5 minutes later)
Okay, latin.bin and katakana.bin files are ready, I should add some codepage stuff in my text display code now !

(1 hour later)
data/images/20160203_font_test3_katakana_text_and_menu.png
Finally some exotic language displayed !
I lost 30 minutes in wondering why application wasn't
initially showing Japanese characters ... and it appears that
I was "cleaning up" the comment string by replacing exotic
character into space character
"It wasn't a bug, it was a feature "

It made sense when Japanese character wasn't displayed
correctly, but it was time to remove this string cleanup code.


Job complete ? Well, there are still some details to fix, and this is what take the most of time when developing software, but at least the boring part of the job is done :]



As a reward for me, I can now set save comment to a cool one when importing save data and that comment isn't defined.
I initially used a dummy "ImportData" string, but the holy "セガサターン,シロ" string defined in hitomi2500's ss-save-parser project is definitely better.
data/images/20160203_save_default_comment.png
セガサターン,シロ !


Last but not least : finishing this main quest opens doors to firmware translation side quest !
Well, Japanese translation would be ugly, because kanji characters are not available (just old-looking katakana characters : this article title would look like ザレージフォックスジャンプオーバーザブラウンドッグ on firmware :D), and on software side, handling multi byte characters is a real mess, so I don't want to do make a Japanese version of Pseudo Saturn Kai and Save Game Manager.

However, translation for European languages is now technically possible
Well, "possible" doesn't means I will do this right now  This is a low priority job, so I will do it when I will have enough time and motivation for it.

BTW, in order to test if exotic character display was OK on Pseudo Saturn, I tried to print "beta" glyph (defined in European codepage) on the title bar :
data/images/20160211_pskai_main_menu.png
Pseudo Saturn Kai main menu
Without any surprises (text display routines are common between Pseudo Saturn Kai and Save Data Manager), it works !
I can finally indicate that this is a beta version ^^





Edit : oops, the feature to inform me when a comment is posted seems (once again) not to work, and consequently I didn't noticed new comments :/

Here are the comments I replied so far. Since I'm late, they deserve to be re-posted here :

I am really happy to hear that you don't have cancer anymore!! Congratulations! I wish you a really long and happy life! :)     Anon

@Anon : It doesn't guarantee I'm cancer free !
CT scans indicate that there is no evidence of cancer relapse, but CT scan has limits (it doesn't shows millimeter-order tumors), so I continue regular cancer screening and taking care in everyday life

--------------
Hey @Spawn, could you upload those Saturn saves you have on your PC?
I have been looking for an archive of Saturn game saves for months now and I haven't found any.
Maybe with the save states I have, and your saves I can upload them somewhere and start an archive? I'll give you credit for it of course.     ReyVGM
@ReyVGM : Thank you for sharing your saves on segaxtreme !
They will be available for everybody on Save Data Manager, so that it will be easy to import on Saturn or emulator.

--------------
best wishes for 2016 !!!     vbt
@vbt : Thank you ! Best wishes for 2016 too !!!
Well, we are already in February, sorry ^^;


--------------
It's great to see your active development, cause my saturn projects are almost in hibernation today (i'm a bit in a time trouble). And i'm glad my code is of some use.
Good luck!     hitomi2500
@hitomi2500 : Hehe, I got some free time until end of March ... I don't know what it will be after that, but what's sure is that I will continue to have fun with my Saturn  Please keep continuing the good work too :]

--------------

stunned and confused

last night I went on a sega saturn nostalgia trip.

I had to catch up with 8+ years @ segaxtreme.
I found antime's usbcart, hitomi2500's fork, hitomi2500's new "wasca" and your stuff here.

sounds great ...
I need to dig up my saturn and finally start something.
I have a usb-cart from 2006 from pinchy (he seems inactive since 2007).     seal1
@seal1 : Hey, I remember seeing your nick in segaxtreme thread about yano/Freewing transfer device (I built a Freewing device at that time).
It was a long time before I made my first USB dev cart ... nostalgia.
Many members forgot about Saturn and left segaxtreme, but luckily vbt and antime are still active.
You're always welcome on segaxtreme, even if it just to say "hi"

Read more The quick lazy brown dog jumps over the foxes ?

Merry New Year !

Written by cafealpha 6 comments
Classified in : PPCenter, Saturn, Mylife Tags : Christmas, Saturn, kai, beta, development, testing
I sometimes check this blog in order to verify if there are some interesting news, without success, and then I realized that I am the only author of this blog (), hence though I should write something here in order not to deceive my countless fans.

So, sorry for the lack of news recently, I was busy with Saturn things, but also in deciphering Christmas presents lists ... and now busy in tidying up lego blocks and trashing used wrapping paper

I hope you're spending nice Christmas vacations ! I'm a little late for wishing you a merry Christmas ( ), and a little early to wish you an Happy New Year ( ), hence the broken title


So what's new with my Saturn projects ? In fact, nothing worth writing an article, but here is a short (and probably incomplete) list anyway :

 1. SD Memory Cart : when overwriting save on Saturn side, keep the old save file on SD card (but remove it from saves list, of course).
Basically, it automatically keeps old saves somewhere on SD card, so that it is possible to revert to a previous state. I hope Saturn RPG fans will like

 2. USB dev cart mode : when USB cable is detected, allow to unlock CD-ROM and then prompt for data upload.
This feature allows to makes tests on Saturn main program using data from CD-ROM without the need of installing a modchip, and without the need of burning a CD-R on each program changes
Special thanks to Shazz for reminding me to add such a feature, antime for wiring FTDI's PWREN signal to Saturn (it was unused for 3 years, but it was useful in order to detect USB connection, and hence hiding USB dev mode for normal users not using USB), and of course CyberWarriorX and jhl for Pseudo Saturn

 3. PCB changes : basically removing few debug "in case of" stuff, and also moving SD card socket in order to fit better with cartridge shell.
It is still WIP, but plan to send the updated gerber to the PCB house on next month.

 4. Few changes to the sources of yabause I'm using for testing my projects.
Especially, I added position save/restore and keyboard shortcut to each debug dialogs ... long time I wanted to do this

 5. In Backup Data Manager, some additions to "Import" feature : backup data dump file read and parse was done a while (4-5 months ?) ago, I just added something to select where (Internal Memory or Cartridge or Floppy) to save selected saves, and plan to port save data extract algorithm next month or so
Thanks to hitomi2500's ss-save-parser project, such "Import" feature is easy to integrate in my Backup Data Manager project


Last but not least, I also want to thank Stac user on Assembler forums
He's doing good job in beta testing, and thanks to his reactivity and good communication, project is progressing faster than I was expecting.
Beta testing is far from being a funny job, especially with me as main programmer (), and when testing requires to play games (this requires time, and bugs on my Saturn project may lead to loose of progress in game), so once more, thank you 

Read more Merry New Year !

Firmware development - will it work somewhere else than in my dev Saturn ?

Written by cafealpha 6 comments
Classified in : Saturn, Programming, Mylife Tags : saturn, memcart, kai, beta, video
I'm still developing the firmware for my custom cartridge, and it worked enough so that I could take a video of it running on real hardware


data/images/20151028_pskai_sdcard_test.jpg
Click on the image in order to see its youtube video.
You can also click here in order to download .avi file.


Video may be blurry, so here are some additional pictures :
data/images/20151029_testcart_usb.tb.jpg
USB & SD card cartridge.
This is the cartridge I mainly use for testing on real hardware
(USB port is used for debug purposes)


data/images/20151029_testcart_beta.tb.jpg
SD card cartridge for Saturn.
This is the version I plan to release in the case development continues well


data/images/20151029_flatcube_from_sd_card.tb.jpg
Flatcube demo loaded and executed from SD card.
This picture is here in order to put a link to the homepage of flatcube demo author


And last but not least, many thanks to beta testers for their cooperation and patience

data/images/20151028_pskai_bt2_saturn.tb.jpg
Beta cartridge loaded on modded Japanese Saturn.
Thanks to
MeteK for beta testing and taking this picture
Last week, MeteK reported that basic SD card access worked on his Saturn

data/images/20151028_pskai_bt1_screen.tb.jpg
Pseudo Saturn Kai System Information screen.
Thanks to Alexandre Souza from www.tabalabs.com.br
for beta testing and taking this picture 

data/images/20151112_tabajaralabs_noshell.tb.jpg - data/images/20151112_tabajaralabs_shell.tb.jpg
Additional pictures from Alexandre : cartridge without, then with shell.
Alexandre is currently trying to 3D print cartridge shells. Please do your best


I still can't getting things working on Alexandre side : SD card doesn't respond to SPI requests from Saturn via my cartridge ... I suppose this is due to a bad hardware design on cartridge side, but I still need to investigate about this ...

Edit (2015/11/11) : it is fixed and verified here
Fortunately, it was not an hardware error, but a firmware (SH-2 side) error.
About details : I access SD card by using SPI protocol, and the reason some cards couldn't be initialized was due to the fact that commands were incorrectly received by SD card.
Sending a dummy synchronization byte before sending command to SD card fixed the problem.
data/images/20151111_sdtest_satlink.tb.png
Log taken from my dev Saturn : first half is SD card initialization sequence,
and second half is data blocks reading.

data/images/20151111_sdtest_yabause_log.tb.png
It works under custom build of yabause too
Thanks to this, most of the tests could be done on PC,
which saved a lot of development time.
Thank you yabause team



After doing some testings on my side, I could verify that :
 - SD card access works fine on all the 6 Saturns I own
 - Some SD card can't be accessed via my cartridge, while the others work fine. (← fixed)

data/images/20151102_test_saturns.tb.jpg
Saturns used for testing.
The one on the top of the picture is my good old dev unit

data/images/20151102_test_sd_ng.tb.jpg
SD card that aren't recognized by my cartridge.
Edit (2015/11/11) : I fixed firmware and 32GB micro SD card above was recognized
(I didn't tested with the other SD cards, but I suppose it is now OK too.)

data/images/20151102_test_sd_ok.tb.jpg
SD cards that were correctly recognized by my cartridge.



To conclude this article, I have to say than this project doesn't make exception to Muphy's Law :
Anything that can go wrong will go wrong

Beginning of year (2015) : Virtual Memory Cart related debug. After verifying that it has the chance to work, I temporary stop its development, mostly because I was bored to continue development of this feature ^^;
Until this summer : a lot of struggle with Pseudo Saturn 0.831/0.832/flasher/etc related things.
September : SD card library related troubles. Try to get a solution by using -and trying to fix- thinfat32 library. Using fat_io_lib was a better choice, thank you hitomi2500 for your advice about this library
October ~ Now : Troubles with SD card on Alexandre side ... will work if another SD card is used (currently waiting for test results) ?

Anything that can go wrong will go wrong
But at least, beta cartridge half works, so I can safely continue its development


I plan to release the first batch of cartridges with SD card included, so that there won't be incompatibility issues on user side.
As my project is a low cost one, and also because it won't load commercial games, I plan to sell the cartridges with low cost (8GB or maybe 16GB) SD cards.
Edit (2015/11/11) : SD card compatibility is likely to be fixed, and as far as possible I don't want to do SD card stock management/etc, I will probably release cartridges only.
There still no release date to announce, because some fixes/features addition/testing/etc is remaining, and also because I mostly do this for fun

Read more Firmware development - will it work somewhere else than in my dev Saturn ?

libfat : maybe you need to loose some weight :D

Written by cafealpha 3 comments
Classified in : Saturn, Programming, Mylife Tags : saturn, memcart, kai, fat32, library
So as you may know, I'm (still) developing the hardware/firmware/everything else for my SD card-based Saturn Cartridge.

One the main module for the firmware of this cartridge is File System access library.

As you may know, there are many File System existing in real worls : FAT, NTFS, exFat, ReiserFs, etc.
And in my project, I choose to use FAT32 in order to access files and directories.
Why FAT32 ? Just because it offers the best rate of advantages vs disadvantages in my project scope.
Advantages :
 - Available everywhere, from Windows 98 to every Linux distribution.
 - Detailed (?) specs available on MS website, lots of free source code available elsewhere.
 - Little resource requirement (at least 1~2KB of work RAM, and few tenths of program memory for code).
Disadvantages :
 - A little hard to understand why internal data structure is so messy ... more understandable after reading FAT16 specs.
 - Complex Long FileName support, and patented by MS too.
 - Write performances may be poor under certain conditions. Example : fragmented file system + large partition size.
 - Free source code available everywhere is sometimes buggy / bloated.

At the very beginning of the project (nearly 4 years ago !), I choose DharmaniTech FAT32 library , and it was easy to use, so that I could read files without many problems on software development side.
But it only supports <512 bytes file write, and a lot of advanced features weren't available, so I then switched to thinfat32 library .
Thinfat32 library was a pain to adapt to Saturn, and some advanced features weren't available, or didn't worked, so I rapidly switched to libfat .

Libfat is great, works without any major trouble, and even supports FAT16
So I kept libfat as FAT32 library for around 3 years.
But one problem of libfat is ... its weight (<- yeah, I wanted to put this pun somewhere ).
 - Sector caching is cool, but library doesn't work without it, so we need at least 8KB of work RAM for it.
 - Source code is very complete, but also very complex, so unless you aren't a talented developer, so can't customize it easily ... and unfortunately I'm not a talented developer
 - Library supports nearly everything for file/directory manipulation, so size after compilation is a little big (around 50KB IIRC).

You may ask : "why complaining about few KBs of RAM ? Saturn got plenty of RAM to use !"
Well, yes and no. My project (on game game backup from/to SD card) of resources :
 - Work RAM : 24KB ... not contiguous : 16KB+8KB
 - Program memory : work RAM, or cartridge ROM (1MB). I choose to run program directly from ROM.

And, in a (maybe) future hardware project, I may need to port FAT32 library to microcontroller, so few resources will be mandatory.

So last month, I dropped libfat, and started to look for a simpler & better FAT32 library.
When selecting a starting point for development, my choice pointed to thinfat32, not because it was perfect, but because I had some experience with it in the past, so that I know where I should start to fix things

First, I dropped some features I don't need :
 - Long FileName when writing files : LFN can be read, but only 8.3 files can be written ... it's enough for my needs
 - Long path : changed maximum size for file path from MAX_PATH (260 bytes) to 96 bytes ... it's enough for me, and saves some RAM.
 - Only one file open at once : it saves some RAM.
 - Other ? (don't remember)

Then started to fix/debug/etc the library :
 - Big/low endian support ... Saturn and PC endianness is different
 - Directory creation fix.
 - File creation in non-root directory fix.
 - Added test bench under windows ... there was one stub in original sources, but with very limited test range ...
 - Better log support.
 - FS info support.
 - FAT16 support ... not working yet (), but at least it works better than in original sources
 - File append fix ... original sources support file append ... only in the case source file is multiple of 512 bytes ... this bug took me many time to spot
 - Clock support when setting file timestamp ... original sources supported only fixed (dummy) timestamp.
 - File listing within a directory.
 - Source code cleanup.
 - MANY fixes.

Complaints about thinfat32 library may look severe, but I really thanks its author to publicly release it !
Thanks to him, I know have something that fits my needs, and have a deeper knowledge regarding FAT32 structure

Last but not least, here are some screenshots :
data/images/20151021_thinfat_testprog.tb.png
Test program contents :
In this screenshot, I first create one directory,
then write 3 files inside it.


data/images/20151021_thinfat_testbench.tb.png
Test program execution - just in order to show
a command line screenshot on my blog

data/images/20151021_thinfat_testresults.tb.png
Result in (virtual) test drive :
one directory and 3 files are created !

This is just the beginning of the development : I still need to test under Saturn emulator (yabause ), then on real hardware (Saturn + custom cartridge ).

Read more libfat : maybe you need to loose some weight :D

Shining Force 3 works !

Written by cafealpha 10 comments
Classified in : Saturn, Programming, Mylife Tags : saturn, memcart, kai, shining force
Additional testing of my test cartridge showed that Shining Force scenario 3 was "working" well when using memory expander hack

By "working", I mean that game was working fine (no freeze, black screen whatever) while looking for save data on SD card.
I didn't verified if save data was consistent or not, this can be verified and eventually fixed later

data/images/20150915_sforce39.tb.png
Game is looking for SFORCE39 save data ?!

Logs show that game is looking for SFORCE31 save data, which is normal, but after that it is looking for SFORCE39 save data ?!?
I suppose this is some debug stuff that was left as-is, or maybe something for game beta testers ? Please let me know if you have more informations about this


It "works", but there's still a lot of code to write/cleanup/etc before release !
At first, I should add some "log to SD card" feature, this may be useful when we will enter beta testing stage

BTW, I'm looking for a name for my "SD card-based memory cart for Saturn" project.
The only requirement is that the name needs to be easy to remember. Suggestions are welcome


Edit (2016/02/12) : nearly 5 months after, everything works more or less so that I took a couple of videos of Pseudo Saturn Kai in action
data/images/20160212_xzibit_video.jpg
Video in video ?

More next week month

Edit (2016/03/16) : took longer than expected to add subtitles to the videos, but everything is finally ready. Thank you for your patience

data/images/20160316_shining_force_3_sd_memcart.jpg
Shining Force 3 Scenario 1 SD Memory Cart test
Game is accessing save data from cartridge's SD card. It works



data/images/20160316_shining_force_3_sd_memcart_makingof.jpg
Shining Force 3 Scenario 1 SD Memory Cart test (Making Of)
Same as above, with also log messages display on my dev laptop

Read more Shining Force 3 works !

Rss feed of the tag