{"id":1454,"date":"2020-05-13T00:49:19","date_gmt":"2020-05-12T23:49:19","guid":{"rendered":"http:\/\/www.fenarinarsa.com\/?p=1454"},"modified":"2023-03-20T10:25:24","modified_gmt":"2023-03-20T09:25:24","slug":"fast-ym-file-format","status":"publish","type":"post","link":"https:\/\/www.fenarinarsa.com\/?p=1454","title":{"rendered":"Fast YM file format"},"content":{"rendered":"\n<p class=\"has-small-font-size\"><em>The FYM tools are now <a href=\"https:\/\/github.com\/fenarinarsa\/FYM\">available on GitHub<\/a><\/em> (source code + binaries). v05 released on 17\/05\/2020!<\/p>\n\n\n\n<p>When I started to work on <a href=\"https:\/\/www.fenarinarsa.com\/?p=1229\">Latecomer<\/a>, the first problem I tried to solve was getting a music.<\/p>\n\n\n\n<p>Fortunately, Grouik\/FT did some <a href=\"http:\/\/fr3nch.t0uch.free.fr\/NSCT\/NSCT.html\">amazing work back in 2015<\/a> by taking Atari ST songs and replaying them on the <a href=\"https:\/\/en.wikipedia.org\/wiki\/Mockingboard\">Mockingboard<\/a>. That&rsquo;s actually what convinced me to try to do demos on Apple II: because I discovered that we could play real tunes.<\/p>\n\n\n\n<figure class=\"wp-block-embed-youtube aligncenter wp-block-embed is-type-video is-provider-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<div class=\"ast-oembed-container\" style=\"height: 100%;\"><iframe loading=\"lazy\" title=\"(Not So) Cheap Tunes\" width=\"500\" height=\"281\" src=\"https:\/\/www.youtube.com\/embed\/KOtXepnIioE?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/div>\n<\/div><\/figure>\n\n\n\n<p>The Mockingboard is a sound card released in 1981 for all Apple IIs (except the IIc of course). It consists mostly of two 6522 VIAs that are connected to two AY-3-8310 chips (for left\/right channels) running at system frequency (~1Mhz). The VIAs are very important too because both expose two timers that can trigger the 6502 interrupt&#8230;<br>A timer is very convenient to play music, and the Apple II doesn&rsquo;t have any.<\/p>\n\n\n\n<p>I will talk about those VIAs later because they&rsquo;re why a Mockingboard is mandatory to make \u00ab\u00a0modern\u00a0\u00bb Apple II effects.<\/p>\n\n\n\n<p>The main issue in order to play music on Apple II is&#8230; well, to have a player. And there&rsquo;s not a lot of choice, because all computers that had AY or YM soundchips were based on Z80 or 68000 &#8211; except for the Oric.<br>On the opposite side, the other widely available other 6502 computer is the C64, and it has its own SID chipset.<\/p>\n\n\n\n<p>That leaves us with almost nothing, so recently people started to come with their own answers.<\/p>\n\n\n\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_82_1 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/www.fenarinarsa.com\/?p=1454\/#PT3_Vortex_player\" >PT3 (Vortex) player<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/www.fenarinarsa.com\/?p=1454\/#ARKOS_Tracker_2_CHPNSFX\" >ARKOS Tracker 2 \/ CHPNSFX<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/www.fenarinarsa.com\/?p=1454\/#YM_Format\" >YM Format<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/www.fenarinarsa.com\/?p=1454\/#MYM_Format\" >MYM Format<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/www.fenarinarsa.com\/?p=1454\/#FYM_format\" >FYM format<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/www.fenarinarsa.com\/?p=1454\/#YM%3EFYM_Converter\" >YM&gt;FYM Converter<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/www.fenarinarsa.com\/?p=1454\/#FYM_Player\" >FYM Player<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/www.fenarinarsa.com\/?p=1454\/#FYM_File_Format\" >FYM File Format<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/www.fenarinarsa.com\/?p=1454\/#PYM_format\" >PYM format<\/a><\/li><\/ul><\/nav><\/div>\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"PT3_Vortex_player\"><\/span>PT3 (Vortex) player<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>In 2019, deater released a <a href=\"https:\/\/demozoo.org\/productions\/272473\/\">PT3 (ProTracker) player<\/a> for Apple II&#8230; and French Touch did also release a demo with <a href=\"http:\/\/fr3nch.t0uch.free.fr\/OMT\/OMT.html\">another PT3 player<\/a>.<\/p>\n\n\n\n<p> It&rsquo;s a very common format in the <a href=\"https:\/\/zxart.ee\/eng\/music\/\">ZX Spectrum demoscene<\/a> and it has many advantages.<\/p>\n\n\n\n<p>First, it has a compatible Windows Tracker, the <a href=\"https:\/\/bulba.untergrund.net\/vortex_e.htm\">Vortex tracker<\/a>, that can handle dual AY setups with 6 channels, because such a card is available on ZX Spectrum.<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img decoding=\"async\" src=\"https:\/\/3.bp.blogspot.com\/-qI-GCx6HT38\/UvPVxihScKI\/AAAAAAAAIHA\/G7rGyttUVBo\/s1600\/vortextracker2.patterns.png\" alt=\"Ultimate Music Tracker Base: VORTEX TRACKER II\"\/><figcaption>Vortex Tracker in 3-channels setup<\/figcaption><\/figure><\/div>\n\n\n\n<p>Next, you can setup the chipset frequency in the tracker. It means you can compose for the 1Mhz Mockingboard, while the ZX Spectrum&rsquo;s AY runs at 1.79Mhz (all the sound frequencies are derived from the main frequency).<\/p>\n\n\n\n<p>And of course, it&rsquo;s a very compact format, so it fits into the Apple II memory nicely.<\/p>\n\n\n\n<p>The drawback is that the PT3 format is poorly documented. French Touch&rsquo;s 6502 player is actually a conversion of the Z80 player while deater&rsquo;s was written from scratch.<\/p>\n\n\n\n<p>It takes a lot of CPU time and the execution time can vary greatly from a frame to another. I other word, if you want a really fast player with a nearly-fixed execution time, it won&rsquo;t fit.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"ARKOS_Tracker_2_CHPNSFX\"><\/span>ARKOS Tracker 2 \/ CHPNSFX<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Those are French Touch&rsquo;s tests of porting the player for the <a href=\"http:\/\/fr3nch.t0uch.free.fr\/FKF\/FKF.html\">ARKOS Tracker 2<\/a> and <a href=\"http:\/\/cngsoft.no-ip.org\/chipnsfx.htm\">CHPNSFX tracker<\/a> formats.<\/p>\n\n\n\n<p>The CHPNSFX is a very lightweight format so it can be useful in some cases. However you need to find a composer that knows how to use the tracker.<\/p>\n\n\n\n<p>The ARKOS Tracker can do soundtrack and export them as RAW, that&rsquo;s what Grouik implemented with a lot of tricks to replay it. Of course the 6502 is used at 100% when playing the tune \ud83d\ude00<\/p>\n\n\n\n<figure class=\"wp-block-embed-youtube aligncenter wp-block-embed is-type-video is-provider-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<div class=\"ast-oembed-container\" style=\"height: 100%;\"><iframe loading=\"lazy\" title=\"FAKE OFF\" width=\"500\" height=\"281\" src=\"https:\/\/www.youtube.com\/embed\/N1UoIqVy-eI?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/div>\n<\/div><figcaption>Don&rsquo;t try this at home<\/figcaption><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"YM_Format\"><\/span>YM Format<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>The YM format was the first one to be played on the Mockingboard. It&rsquo;s a format created by <a href=\"http:\/\/leonard.oxg.free.fr\/ymformat.html\">Leonard\/Oxygene<\/a> for the first Windows AY emulator ever (ST-Sound), and it&rsquo;s as simple as it&rsquo;s well documented.<\/p>\n\n\n\n<p>Basically, it&rsquo;s a dump of all AY registers for every video frame refresh which is the packed in LZH. There&rsquo;s some more things to know about special effects and how the registers are written into the files, but that&rsquo;s how it works.<\/p>\n\n\n\n<p>Of course it makes a very, very good candidate for an Apple II player since we can dump a YM file from quite any other AY format.<\/p>\n\n\n\n<p>(you can use <a href=\"https:\/\/bulba.untergrund.net\/emulator_e.htm\">AY_Emul<\/a> to convert any supported format to YM)<\/p>\n\n\n\n<p>The main issue with the YM format is the LZH compression, because it&rsquo;s not a format fit for old platforms. You can either unpack it before playing,  or unpack it on the fly (which is impossible for a 6502 at 1Mhz).<\/p>\n\n\n\n<p>For a 3mn song, the unpacked data takes around 126kB of RAM&#8230; so well, we can&rsquo;t use it directly.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"MYM_Format\"><\/span>MYM Format<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>The <a href=\"http:\/\/osdk.org\/index.php?page=documentation&amp;subpage=ym2mym\">MYM format<\/a> is basically the YM format with a different compression scheme. It&rsquo;s been developed by Marq for MSX and ported to 6502 by <a href=\"http:\/\/www.defence-force.org\/computing\/oric\/dbug\/index.htm\">DBUG\/Defence Force<\/a> for Oric. It&rsquo;s tailored for low-end platforms.<\/p>\n\n\n\n<p>It packs registers together by discarding bits that are unused by the AY (<a href=\"http:\/\/osdk.org\/index.php?page=articles&amp;ref=ART5\">read the description here<\/a>).<br>The player then unpacks data on-the-fly and rebuilds the register data from the unpacked stream.<\/p>\n\n\n\n<p>I don&rsquo;t really know how fast the player is. The source code of the converter is <a href=\"http:\/\/miniserve.defence-force.org\/viewvc\/projet1\/public\/pc\/tools\/osdk\/main\/Ym2Mym\/\">available here<\/a>.<\/p>\n\n\n\n<p>Grouik used the MYM format for his 6-channels player in <a href=\"http:\/\/fr3nch.t0uch.free.fr\/PNO\/PNO.html\">\u00ab\u00a0Pure Noise\u00a0\u00bb (2016)<\/a>.<\/p>\n\n\n\n<figure class=\"wp-block-embed-youtube aligncenter wp-block-embed is-type-video is-provider-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<div class=\"ast-oembed-container\" style=\"height: 100%;\"><iframe loading=\"lazy\" title=\"Pure Noise\" width=\"500\" height=\"281\" src=\"https:\/\/www.youtube.com\/embed\/njBpe33kbAg?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/div>\n<\/div><figcaption>6-channels MYM replay<\/figcaption><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"FYM_format\"><\/span>FYM format<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>The YM format and MYM format seemed good, but I was bothered by the fact they take a \u00ab\u00a0dumb\u00a0\u00bb approach of packing the data without making any sense of it, and without looking for redundancy first.<\/p>\n\n\n\n<p>And we&rsquo;re talking about chiptune music, which is generated by a tracker, which works on a sequence\/pattern basis. So OF COURSE there should be some form of redundancy in the final register dump.<\/p>\n\n\n\n<p>After many tests and iterations, I ended with the FYM file format, which is a very light file format that can easily be unpacked on the fly on a 6502 and takes the minimum possible memory. It&rsquo;s also byte-based, where MYM needs bit-level operations.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"YM%3EFYM_Converter\"><\/span>YM&gt;FYM Converter<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>The converter takes a YM file as input (YM3 to YM6). No special effect (digi-drums, SID&#8230;) is handled.<\/p>\n\n\n\n<p>Then, it looks for redundancy, based on a certain pattern length.<\/p>\n\n\n\n<p>Let say we start with a length of 16, the algorithm is:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Take all 14 registers&rsquo; dump individually for the whole tune<\/li><li>Break them into patterns of 16 bytes (16 frames)<\/li><li>Look for redundancy in all 16-bytes patterns, regardless of registers<\/li><li>Pack all individual deduplicated patterns using Packbits (byte-based RLE)<\/li><li>Reorganize all packed patterns in a way they never cross a 256-bytes boundary (better for the 6502 player) using a bin packing algorithm<\/li><li>Replace the original dump by 16 bits pointers to the packed patterns<\/li><\/ul>\n\n\n\n<p>Eventually, for 14 registers * 16 bytes (224 bytes), we get only 14 * 2 bytes (28 bytes), so the original dump is reduced by 8!<\/p>\n\n\n\n<p>But of course you need to add the packed patterns data. And the smaller the pattern&rsquo;s length is, the more pointers you need to add in the file. On the other hand, a large length will result in less deduplication.<\/p>\n\n\n\n<p>In order to find the correct balance, I simply use brute force and try any length between 16 and 128. Actually every resulting \u00ab\u00a0best length\u00a0\u00bb was a power of 2 and fit perfectly with the rhythm of the tune. COULD YOU BELIEVE IT.<\/p>\n\n\n\n<figure class=\"wp-block-embed-youtube aligncenter wp-block-embed is-type-video is-provider-youtube wp-embed-aspect-4-3 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<div class=\"ast-oembed-container\" style=\"height: 100%;\"><iframe loading=\"lazy\" title=\"YM to FYM converter\" width=\"500\" height=\"375\" src=\"https:\/\/www.youtube.com\/embed\/QUy2q2I5m7o?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/div>\n<\/div><figcaption>Rollout2.ym = 3,305 bytes<br>(unpacked) = 170,488 bytes<br>MYM = 11,740 bytes<br>FYM = 13,568 bytes<\/figcaption><\/figure>\n\n\n\n<p>In addition, the final file can also be packed using a 6502-friendly compression scheme.<br>For rollout2.ym (above example), the final results with LZ4 are:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>MYM+LZ4 &#8211; 8,722 bytes<\/li><li>FYM+LZ4 &#8211; 5,481 bytes<\/li><\/ul>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"FYM_Player\"><\/span>FYM Player<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>The first thing the player has to do is to relocate the pointers (the \u00ab\u00a0partition\u00a0\u00bb) because the pointers are actually offsets from the start of the file.<br>On Apple II there&rsquo;s only need to change the high byte value of each pointer because the file is always located at a 0x0100 boundary.<\/p>\n\n\n\n<p>The length of the patterns is given at the start of the file, and it never changes. That allows the player to stop unpacking patterns and fetch another batch of pointers based on length instead of looking for the 0x80 end marker used in packbits (more bytes saved!).<br>Then it has to maintain data for unpacking 14 parallel patterns, which is not a lot of work and very 6502-friendly.<\/p>\n\n\n\n<p>The only drawback is when the player reaches the end of the current patterns, it must fetch 14 pointers and reset 14 packbits data, which is the worst case you can get. It may be avoided by preparing that data beforehand in a progressive way.<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"960\" height=\"720\" src=\"https:\/\/www.fenarinarsa.com\/wp-content\/vlcsnap-2020-05-13-00h19m55s722.png\" alt=\"\" class=\"wp-image-1497\" srcset=\"https:\/\/www.fenarinarsa.com\/wp-content\/vlcsnap-2020-05-13-00h19m55s722.png 960w, https:\/\/www.fenarinarsa.com\/wp-content\/vlcsnap-2020-05-13-00h19m55s722-300x225.png 300w, https:\/\/www.fenarinarsa.com\/wp-content\/vlcsnap-2020-05-13-00h19m55s722-768x576.png 768w\" sizes=\"auto, (max-width: 960px) 100vw, 960px\" \/><figcaption>The white bar is the CPU time taken by the final player<br>(average case, around 20 lines, 7% on a PAL config).<br>Yes you can see that on Apple II. Remember it&rsquo;s a 1Mhz 6502.<\/figcaption><\/figure><\/div>\n\n\n\n<p>The player was based on French Touch&rsquo;s first YM player, with additional data displayed. The reverb is due to the fact it&rsquo;s not possible to program the two AYs at exactly the same time.<\/p>\n\n\n\n<p>The source code for the generator (C#) and the 6502 player (+binaries) is <a href=\"https:\/\/github.com\/fenarinarsa\/FYM\">available on GitHub<\/a>.<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Registers (line 1) = register number (00, 11, 22&#8230; DD)<\/li><li>Registers (line 2) = register value<\/li><li>Patterns ptr (line 1) = High byte of current pattern pointer for this register<\/li><li>Patterns ptr (line 2) = Low byte of current pattern pointer for this register<\/li><li>Partition ptr = Pointer to the current set of 14 patterns pointer, followed by the remaining length to decode<\/li><li>CPU = cycles used by the stereo player (complete frame: 20280 cycles in PAL, 17030 cycles in NTSC)<\/li><\/ul>\n\n\n\n<p>You can see that the \u00ab\u00a0sequence\u00a0\u00bb pointers progress only when the registers change, which is of course a consequence of the Packbits\/RLE scheme.<\/p>\n\n\n\n<figure class=\"wp-block-embed-youtube aligncenter wp-block-embed is-type-video is-provider-youtube wp-embed-aspect-4-3 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<div class=\"ast-oembed-container\" style=\"height: 100%;\"><iframe loading=\"lazy\" title=\"FYM player v05 - YM player on Apple II\" width=\"500\" height=\"375\" src=\"https:\/\/www.youtube.com\/embed\/QmqyOMQcxWE?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe><\/div>\n<\/div><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"FYM_File_Format\"><\/span>FYM File Format<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>Given the explanations above, you can easily make your own file format, but here&rsquo;s the one I used for v05:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>1 byte = 0x00 (reserved)<\/li><li>1 byte = Patterns length<\/li><li>2 bytes = Offset from the start of file to the start of partition<\/li><li>n bytes = packed register patterns<\/li><li>n times * [ 14 bytes (HIGH BYTE) + 14 bytes (LOW BYTE)  = offset to the packed pattern for each register ] = partition<\/li><li>2 bytes = 0x0000, end of partition<\/li><\/ul>\n\n\n\n<p>This format can still be improved, because it happens that in some tunes, some registers are not used, so it&rsquo;s useless to save them (deleting 1 register reduce the partition size by 7%).<\/p>\n\n\n\n<p>One of the advantage of this file format is that you can easily rework any tune, since the patterns&rsquo; length often match the logical length of the original tracker patterns.<\/p>\n\n\n\n<p>That what I did in Latecomer, I cut out a part of the song (with the authorization of Big Alec) to make the demo shorter. And it was originally a YM file! It&rsquo;s only a matter of deleting some pointers in the regenerated \u00ab\u00a0partition\u00a0\u00bb. I actually made an array in the generator containing the final partition I wanted. As I was out of time though, I did reduce the partition without regenerating the packed patterns, so the filler between the partition and the data is huge and there is surely unused patterns left. But the final file size is 12kB anyway!<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"PYM_format\"><\/span>PYM format<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Late addition to this article, because I find it interesting.<\/p>\n\n\n\n<p>Grouik\/FT worked on a very similar file format, except that the player is even faster than the FYM one.<\/p>\n\n\n\n<p>It uses basically the same deduplication scheme of FYM for the YM dump, except that he uses a simplified version of Packbots\/RLE that handles only the deduplication case for patterns. So a pattern that has no redundancy like 01 02 03 04 05 06 07&#8230; will always pack as 01 01 01 02 01 03 01 04 01 05 01 06 01 07 (n redundancies of 1 byte) and be twice the size than the original data. On the other hand, the player code is far more simpler because there&rsquo;s only two cases to handle (duplicate byte &amp; load a control byte as new counter) where Packbits has three with a more complex way to handle the control byte. As a result, the PYM player is also more compact and it&rsquo;s far easier to unroll code.<\/p>\n\n\n\n<p>Grouik also used a file format which is more 6502-friendly, with pointers split between high\/low bytes, and a maximum of 256 lines in the main tune map to use 6502&rsquo;s indexed mode. In the end it allows for around 65k frames (enough for 21mn of music) so I guess FYM file format could take advantage of the same formatted pointers list.<\/p>\n\n\n\n<p>PYM is basically simpler and better optimized for 6502, while the player is faster than FYM. The drawback being that the resulting files are around 20% larger than FYM.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The FYM tools are now available on GitHub (source code + binaries). v05 released on 17\/05\/2020! When I started to work on Latecomer, the first problem I tried to solve was getting a music. Fortunately, Grouik\/FT did some amazing work back in 2015 by taking Atari ST songs and replaying them on the Mockingboard. That&rsquo;s &hellip;<\/p>\n<p class=\"read-more\"> <a class=\"\" href=\"https:\/\/www.fenarinarsa.com\/?p=1454\"> <span class=\"screen-reader-text\">Fast YM file format<\/span> Lire la suite\u00a0\u00bb<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"rop_custom_images_group":[],"rop_custom_messages_group":[],"rop_publish_now":"initial","rop_publish_now_accounts":[],"rop_publish_now_history":[],"rop_publish_now_status":"pending","site-sidebar-layout":"default","site-content-layout":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","theme-transparent-header-meta":"default","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","footnotes":""},"categories":[12,9],"tags":[],"class_list":["post-1454","post","type-post","status-publish","format-standard","hentry","category-apple-ii-2","category-demoscene"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.2 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Fast YM file format - Coredump<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.fenarinarsa.com\/?p=1454\" \/>\n<meta property=\"og:locale\" content=\"fr_FR\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Fast YM file format - Coredump\" \/>\n<meta property=\"og:description\" content=\"The FYM tools are now available on GitHub (source code + binaries). v05 released on 17\/05\/2020! When I started to work on Latecomer, the first problem I tried to solve was getting a music. Fortunately, Grouik\/FT did some amazing work back in 2015 by taking Atari ST songs and replaying them on the Mockingboard. That&rsquo;s &hellip; Fast YM file format Lire la suite\u00a0\u00bb\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.fenarinarsa.com\/?p=1454\" \/>\n<meta property=\"og:site_name\" content=\"Coredump\" \/>\n<meta property=\"article:published_time\" content=\"2020-05-12T23:49:19+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-03-20T09:25:24+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/3.bp.blogspot.com\/-qI-GCx6HT38\/UvPVxihScKI\/AAAAAAAAIHA\/G7rGyttUVBo\/s1600\/vortextracker2.patterns.png\" \/>\n<meta name=\"author\" content=\"Cyril Lambin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@fenarinarsa\" \/>\n<meta name=\"twitter:site\" content=\"@fenarinarsa\" \/>\n<meta name=\"twitter:label1\" content=\"\u00c9crit par\" \/>\n\t<meta name=\"twitter:data1\" content=\"Cyril Lambin\" \/>\n\t<meta name=\"twitter:label2\" content=\"Dur\u00e9e de lecture estim\u00e9e\" \/>\n\t<meta name=\"twitter:data2\" content=\"10 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.fenarinarsa.com\/?p=1454#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.fenarinarsa.com\/?p=1454\"},\"author\":{\"name\":\"Cyril Lambin\",\"@id\":\"https:\/\/www.fenarinarsa.com\/#\/schema\/person\/80a26199aabcf306072d581606fb24e4\"},\"headline\":\"Fast YM file format\",\"datePublished\":\"2020-05-12T23:49:19+00:00\",\"dateModified\":\"2023-03-20T09:25:24+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.fenarinarsa.com\/?p=1454\"},\"wordCount\":2043,\"image\":{\"@id\":\"https:\/\/www.fenarinarsa.com\/?p=1454#primaryimage\"},\"thumbnailUrl\":\"https:\/\/3.bp.blogspot.com\/-qI-GCx6HT38\/UvPVxihScKI\/AAAAAAAAIHA\/G7rGyttUVBo\/s1600\/vortextracker2.patterns.png\",\"articleSection\":[\"Apple II\",\"Demoscene\"],\"inLanguage\":\"fr-FR\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.fenarinarsa.com\/?p=1454\",\"url\":\"https:\/\/www.fenarinarsa.com\/?p=1454\",\"name\":\"Fast YM file format - Coredump\",\"isPartOf\":{\"@id\":\"https:\/\/www.fenarinarsa.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.fenarinarsa.com\/?p=1454#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.fenarinarsa.com\/?p=1454#primaryimage\"},\"thumbnailUrl\":\"https:\/\/3.bp.blogspot.com\/-qI-GCx6HT38\/UvPVxihScKI\/AAAAAAAAIHA\/G7rGyttUVBo\/s1600\/vortextracker2.patterns.png\",\"datePublished\":\"2020-05-12T23:49:19+00:00\",\"dateModified\":\"2023-03-20T09:25:24+00:00\",\"author\":{\"@id\":\"https:\/\/www.fenarinarsa.com\/#\/schema\/person\/80a26199aabcf306072d581606fb24e4\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.fenarinarsa.com\/?p=1454#breadcrumb\"},\"inLanguage\":\"fr-FR\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.fenarinarsa.com\/?p=1454\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"fr-FR\",\"@id\":\"https:\/\/www.fenarinarsa.com\/?p=1454#primaryimage\",\"url\":\"https:\/\/3.bp.blogspot.com\/-qI-GCx6HT38\/UvPVxihScKI\/AAAAAAAAIHA\/G7rGyttUVBo\/s1600\/vortextracker2.patterns.png\",\"contentUrl\":\"https:\/\/3.bp.blogspot.com\/-qI-GCx6HT38\/UvPVxihScKI\/AAAAAAAAIHA\/G7rGyttUVBo\/s1600\/vortextracker2.patterns.png\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.fenarinarsa.com\/?p=1454#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Accueil\",\"item\":\"https:\/\/www.fenarinarsa.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Fast YM file format\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.fenarinarsa.com\/#website\",\"url\":\"https:\/\/www.fenarinarsa.com\/\",\"name\":\"Coredump\",\"description\":\"Le blog de Fenarinarsa, le pays du bonheur et des oiseaux qui font cui-cui.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.fenarinarsa.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"fr-FR\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.fenarinarsa.com\/#\/schema\/person\/80a26199aabcf306072d581606fb24e4\",\"name\":\"Cyril Lambin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"fr-FR\",\"@id\":\"https:\/\/secure.gravatar.com\/avatar\/c3c1a506ae7c6720c92de38fcb90b4a4cfd9210af4f94de3604bb07987384f5d?s=96&d=mm&r=g\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/c3c1a506ae7c6720c92de38fcb90b4a4cfd9210af4f94de3604bb07987384f5d?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/c3c1a506ae7c6720c92de38fcb90b4a4cfd9210af4f94de3604bb07987384f5d?s=96&d=mm&r=g\",\"caption\":\"Cyril Lambin\"},\"sameAs\":[\"https:\/\/fenarinarsa.com\"],\"url\":\"https:\/\/www.fenarinarsa.com\/?author=2\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Fast YM file format - Coredump","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.fenarinarsa.com\/?p=1454","og_locale":"fr_FR","og_type":"article","og_title":"Fast YM file format - Coredump","og_description":"The FYM tools are now available on GitHub (source code + binaries). v05 released on 17\/05\/2020! When I started to work on Latecomer, the first problem I tried to solve was getting a music. Fortunately, Grouik\/FT did some amazing work back in 2015 by taking Atari ST songs and replaying them on the Mockingboard. That&rsquo;s &hellip; Fast YM file format Lire la suite\u00a0\u00bb","og_url":"https:\/\/www.fenarinarsa.com\/?p=1454","og_site_name":"Coredump","article_published_time":"2020-05-12T23:49:19+00:00","article_modified_time":"2023-03-20T09:25:24+00:00","og_image":[{"url":"https:\/\/3.bp.blogspot.com\/-qI-GCx6HT38\/UvPVxihScKI\/AAAAAAAAIHA\/G7rGyttUVBo\/s1600\/vortextracker2.patterns.png","type":"","width":"","height":""}],"author":"Cyril Lambin","twitter_card":"summary_large_image","twitter_creator":"@fenarinarsa","twitter_site":"@fenarinarsa","twitter_misc":{"\u00c9crit par":"Cyril Lambin","Dur\u00e9e de lecture estim\u00e9e":"10 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.fenarinarsa.com\/?p=1454#article","isPartOf":{"@id":"https:\/\/www.fenarinarsa.com\/?p=1454"},"author":{"name":"Cyril Lambin","@id":"https:\/\/www.fenarinarsa.com\/#\/schema\/person\/80a26199aabcf306072d581606fb24e4"},"headline":"Fast YM file format","datePublished":"2020-05-12T23:49:19+00:00","dateModified":"2023-03-20T09:25:24+00:00","mainEntityOfPage":{"@id":"https:\/\/www.fenarinarsa.com\/?p=1454"},"wordCount":2043,"image":{"@id":"https:\/\/www.fenarinarsa.com\/?p=1454#primaryimage"},"thumbnailUrl":"https:\/\/3.bp.blogspot.com\/-qI-GCx6HT38\/UvPVxihScKI\/AAAAAAAAIHA\/G7rGyttUVBo\/s1600\/vortextracker2.patterns.png","articleSection":["Apple II","Demoscene"],"inLanguage":"fr-FR"},{"@type":"WebPage","@id":"https:\/\/www.fenarinarsa.com\/?p=1454","url":"https:\/\/www.fenarinarsa.com\/?p=1454","name":"Fast YM file format - Coredump","isPartOf":{"@id":"https:\/\/www.fenarinarsa.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.fenarinarsa.com\/?p=1454#primaryimage"},"image":{"@id":"https:\/\/www.fenarinarsa.com\/?p=1454#primaryimage"},"thumbnailUrl":"https:\/\/3.bp.blogspot.com\/-qI-GCx6HT38\/UvPVxihScKI\/AAAAAAAAIHA\/G7rGyttUVBo\/s1600\/vortextracker2.patterns.png","datePublished":"2020-05-12T23:49:19+00:00","dateModified":"2023-03-20T09:25:24+00:00","author":{"@id":"https:\/\/www.fenarinarsa.com\/#\/schema\/person\/80a26199aabcf306072d581606fb24e4"},"breadcrumb":{"@id":"https:\/\/www.fenarinarsa.com\/?p=1454#breadcrumb"},"inLanguage":"fr-FR","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.fenarinarsa.com\/?p=1454"]}]},{"@type":"ImageObject","inLanguage":"fr-FR","@id":"https:\/\/www.fenarinarsa.com\/?p=1454#primaryimage","url":"https:\/\/3.bp.blogspot.com\/-qI-GCx6HT38\/UvPVxihScKI\/AAAAAAAAIHA\/G7rGyttUVBo\/s1600\/vortextracker2.patterns.png","contentUrl":"https:\/\/3.bp.blogspot.com\/-qI-GCx6HT38\/UvPVxihScKI\/AAAAAAAAIHA\/G7rGyttUVBo\/s1600\/vortextracker2.patterns.png"},{"@type":"BreadcrumbList","@id":"https:\/\/www.fenarinarsa.com\/?p=1454#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Accueil","item":"https:\/\/www.fenarinarsa.com\/"},{"@type":"ListItem","position":2,"name":"Fast YM file format"}]},{"@type":"WebSite","@id":"https:\/\/www.fenarinarsa.com\/#website","url":"https:\/\/www.fenarinarsa.com\/","name":"Coredump","description":"Le blog de Fenarinarsa, le pays du bonheur et des oiseaux qui font cui-cui.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.fenarinarsa.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"fr-FR"},{"@type":"Person","@id":"https:\/\/www.fenarinarsa.com\/#\/schema\/person\/80a26199aabcf306072d581606fb24e4","name":"Cyril Lambin","image":{"@type":"ImageObject","inLanguage":"fr-FR","@id":"https:\/\/secure.gravatar.com\/avatar\/c3c1a506ae7c6720c92de38fcb90b4a4cfd9210af4f94de3604bb07987384f5d?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/c3c1a506ae7c6720c92de38fcb90b4a4cfd9210af4f94de3604bb07987384f5d?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/c3c1a506ae7c6720c92de38fcb90b4a4cfd9210af4f94de3604bb07987384f5d?s=96&d=mm&r=g","caption":"Cyril Lambin"},"sameAs":["https:\/\/fenarinarsa.com"],"url":"https:\/\/www.fenarinarsa.com\/?author=2"}]}},"_links":{"self":[{"href":"https:\/\/www.fenarinarsa.com\/index.php?rest_route=\/wp\/v2\/posts\/1454","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.fenarinarsa.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.fenarinarsa.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.fenarinarsa.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.fenarinarsa.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1454"}],"version-history":[{"count":45,"href":"https:\/\/www.fenarinarsa.com\/index.php?rest_route=\/wp\/v2\/posts\/1454\/revisions"}],"predecessor-version":[{"id":1552,"href":"https:\/\/www.fenarinarsa.com\/index.php?rest_route=\/wp\/v2\/posts\/1454\/revisions\/1552"}],"wp:attachment":[{"href":"https:\/\/www.fenarinarsa.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1454"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.fenarinarsa.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1454"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.fenarinarsa.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1454"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}