How I recorded and decoded POKEY's polynomial counters;Open Pokey Recipe for the curious

Hello

This is a "reprint" of the comments inside "POKEY.sfz", the chipsounds POKEY configuration file. (the original date is March 2009)

******************************************************************************

Was I naive to think that POKEY was just "another TIA", and I spent enough evenings to regret it. Making the breadboard and having Pokey spew some tones was the easy part, but soon realized that various frequencies gave me different patterns (sometimes after retriggereing the SAME note!)

So I tried many things, recorded GB's of tests and had a few ideas.

After lots of trials and loss of sanity:
Here is the FINAL methodology that I used to get usable data:

A)Clock a POKEY with a VERY SLOW clock (I used a +- 10Khz 50/50 PWM signal from a PIC)

B)Set Bit 6 of AUDC1 ( "1.79Mhz" or cpu clock)

C)Play the highest notes you can for each pattern: AUDF1=0 then AUDF1=1, (...), AUDF1=4

D)Record each bit pattern and pass it through our patternfinder util.

Now since the results are NOT the source pattern, but a "resampled" pattern made after skipping 4+AUDF source pulses for each destination pulse (according to the POKEY spec's "MODIFIED FORMULA"), we must derive the actual SOURCE pattern from the results.

The trivial thing to try is this:

void Shuffler(const char*in, char*out, size_t len, size_t shufsize){

    size_t cnt=0;
    for (size_t i=0;i<len;++i){

        out[i] = in[cnt];
        cnt+=shufsize;

        cnt = cnt % len;
    }
}

In this case my recorded AUDF1=0's 4, 9 and 17 bit patterns gave the exact same results as source OR destination (with added offset before looping of course), which kinda proves that AUDF1=0 can give you the full pattern, and that there is magic somewhere (try that with any other random bit pattern for fun). With this source pattern I was able to verify my own "software resampling" using AUDF1=1,2,3... and so forth from my recordings for a match.

4bit poly: 000011101100101

However, and that's where the fun ends, the 5bit pattern was a pain. Patternfinder kept giving me a 62 bit long repeated pattern:

"00000010111000011001010010011101111110100011110011010110110001"

and NOT a 31 bit pattern like I expected!

I noticed that the former 31 bit half of that pattern was always the inverse of the later (whatever point at which you start).A "hint" that the results im getting use some form of inversion. Schematics show that the source pattern passes through the same circuit logic as the "plain square" so it is used as a "please invert previous bit" matrix

SO: Inverting THAT pattern (fill/skip_4 a 31 bit pattern) gave me the following from my 62 pattern: "1101001100000111001000101011110". Which, as with the 4,9,17 case was used to test for the generation of AUDF1=1 and other frequencies.

chipsounds specific:

A few unique oscillators have been made just for this chip.
One and Two Stream AND-ers and One Stream Differentiators were required to emulate the inner process of the POKEY chip.

Invaluable help from "DeRe Atari Chapter 7", and from the official specs.
And for the various Pinouts/Skematics found online.

Last minute addition: Odyssey 2 (P824x) VDC chip!

Building a software product and analyzing hardware are two very important facets of a product like chipsounds. Do mostly the analysis and you have a very techy app that no one understands, do only the former and you miss that that hands on feeling and the smell of printed circuit boards.

So this weekend I took a little break from coding (which is very close to done anyway) And looked at the few consoles in my lot that are not fully analyzed yet (I will add more chips in updates). And decided that chipsounds 1.0 required the Odyssey2.



The interesting thing is that there is no sound-coder-friendly documentation on the P8244/P8245 online yet, so I figured might as well publish everything I've learned so far.

My research consisted in confirming Sören Gust's info from g7kbios.pdf and to go further in trying various combination of tones and timbres that would be possible on the console itself without using IRQ based resynchs and also to check on MESS's source code theory of the "OR"ING of tone and noise, and to check for their claim of a 16 bit LFSR used for the noise channel.

On top of his great commented Bios document, Gust kindly provided the community with example assembler "Hello Worlds" for the console and even an improved the BIOS sound player which I then used to run tests on the "real thing", using a ripped apart Baseball Cart and a 2732 eprom (2k code is mirrored twice)... don't have 2716's at hand yet

The available docs mention a 24 bit shift register that is clocked by only two dividers: 983Hz 3933Hz, without any concrete info on how those two frequencies are achieved by the hardware from the master clocks in the first place.

So what is the main clock divisors for this chip?

Before trying to hack in assembly/eprom, I played with the few cartridges I had, and found out that pressing (and holding) the reset button stalls the CPU, but not the VDC, which keeps playing the current tone (or noise) - very handy indeed.

Long tone segments were recorded by tapping the audio signal at the junction of the main board and the RF box. (a tad bit of LP filtering at this point but nothing heavy)

Various NTSC Recorded pitches (Bidule FFT/PVOC Loudest freq analysis)
1966.572266Hz
983.285583Hz
655.523987Hz
327.762299Hz
163.881409Hz
81.949478Hz

Assuming 1966.572266Hz came from this pattern: 101010101010101010101010 this means a high freq of would be generated from a clock of 3933.144532 (2x) Which tends to confirm the 3933 value. (note: 983 is just 3933/4)

The preliminary docs for the Intel VDC included in Arnim Läuger's
mcs-48.pdf Mentions the 3933 value comes from 15734/4 but nothing else. Looking around for the source of this frequency:

NTSC television scans 525 lines about 29.97* times a second = 15734.25)
SO -> Precise (theoretic) NTSC values would then be:
15734.25/4 = 3933.5625 (high from now on)
15734.25/16 = 983.390625 (low from now on)

(PAL television scans 625 lines * 25 times a second = 15625)
Precise (theoretic) PAL values would then be:
15625/4 = 3906.25
15625/16 = 976.5625
(would love a Videopak machine to confirm)

---------------------------------------------------

Using Gust's improved BIOS player:

We can verify that:

1)DAC is linear, from recordings of a programmed ramp of the 16 volume values

2)Noise is a LFSR-based 16383 bit long repeating pattern.
I haven't yet taken the time to find the exact LFSR formula,
but i've added it as a table.

3)We can indeed generate any looping 24 bit patterns we want:
Example: 0xFFFF00 (2/3 duty)
0x080000 (1/24 duty)
0x001403 (000000000001010000111011 or padded TIA pattern)
Byte Order check with output:
0x010203 (000000010000001000000011)
Recorded output: from left to right starts with:
(?)11000000010000001000000011000000010000001(...)

Listen to those here

4) Possible pitches (using 50% duty as guide):

24 bit
000000000000111111111111:(low)    40.9746Hz
000000000000111111111111:(high)  163.8984Hz

12bits
000000111111000000111111:(low)    81.9492Hz
000000111111000000111111:(high)  327.7969Hz

8bits
000011110000111100001111:(low)   122.9238Hz
000011110000111100001111:(high)  491.6953Hz

6bits
000111000111000111000111:(low)   163.8984Hz
000111000111000111000111:(high)  655.5938Hz

4bits
001100110011001100110011:(low)   245.8477Hz
001100110011001100110011:(high)  983.3906Hz

2bits
010101010101010101010101:(low)   491.6953Hz
010101010101010101010101:(high) 1966.7813Hz

That's all well and good, but what about 24/8=3 bits?
yes that would make another pitch frequency available, 
but could only play 011 and 001 (1/3 and 2/3 duty)

3bits
011011011011011011011011:(low)   327.7969Hz
011011011011011011011011:(high) 1311.1875Hz New freq!!!!


in pitch order:
000000000000111111111111:(low)     40.9746Hz  MIDI:28( E2):  41.20344543
000000111111000000111111:(low)     81.9492Hz  MIDI:40( E3):  82.40689087
000011110000111100001111:(low)    122.9238Hz  MIDI:47( B3): 123.4708252
000000000000111111111111:(high)   163.8984Hz  MIDI:52( E4): 164.8137817
000111000111000111000111:(low)    163.8984Hz  MIDI:52( E4): 164.8137817
001100110011001100110011:(low)    245.8477Hz  MIDI:59( B4): 246.9416504
011011011011011011011011:(low)    327.7969Hz  MIDI:64( E5): 329.6275635
000000111111000000111111:(high)   327.7969Hz  MIDI:64( E5): 329.6275635
010101010101010101010101:(low)    491.6953Hz  MIDI:71( B5): 493.8833008
000011110000111100001111:(high)   491.6953Hz  MIDI:71( B5): 493.8833008
000111000111000111000111:(high)   655.5938Hz  MIDI:76( E6): 659.2551270
001100110011001100110011:(high)   983.3906Hz  MIDI:83( B6): 987.7666016
011011011011011011011011:(high)  1311.1875Hz  MIDI:88( E7): 1318.510254
010101010101010101010101:(high)  1966.7813Hz  MIDI:95( B7): 1975.533203

So there are 10 UNIQUE 50/50 pitches, plus one 2/3 pitch, but LOTS more different _timbres_ using other patterns. The BIOS engineers only included the 10 "different" (to the ear) 50/50 ones...

At least they found a right divider/pattern length to generate a mostly tuned E5 Scale
( -9.6417 to -7.6867 cents off Equal Temperament in NTSC)
(-21.7044 to -19.7494 cents off Equal Temperament in PAL)

Just enough for a PO-LING :)
Back to possible Timbres Lets fill some timbre void shall we?
A pitch is perceived by the repetition of a pattern:

2^24= 16777216 possible timbres for 40.9746Hz and 163.898560Hz
2^12= 4096 possible timbres for 81.951134Hz and 327.796722Hz
2^8 = 256 possible timbres for 122.919807Hz and 491.694153Hz
2^6 = 64 possible timbres for 163.898621Hz and 655.593628Hz
2^4 = 16 possible timbres for 245.850220Hz and 983.390808Hz
2^3 = 8 possible timbres for 327.7969Hz and 1311.1875Hz
2^2 = 4 possible timbres for 491.696228Hz and 1966.781128Hz

case 2^2 can only really do 50/50 since 01 and 10 sounds the same
... in a monophonic console. 00 is silence; 11 is a DC offset click

case 2^4 following the previous logic can only do 25% 50% and 75% duty...

We did not map them all in chipsounds, but this could be added in the future in a more specific oscillator ...

VIC-20 MIDI Interface and Synth Cartridge Prototype

The VIC-20 was my first computer. You can imagine how close and personal I am with the thing. I do feel that its been unjustly historically overshadowed by the computer that followed it, especially in the audio/music scene.

Viznut and PwP's Robotic Liberation was a real blast and it probably revitalized the scene a bit.

But what we need is a musician friendly way to tackle the beast!
Three years or so ago I requested the help of François Leveillé (aka eslapion) on the Denial forum, a VERY nice electronics guru to help me construct a VIC MIDI Interface like the one HERE . So that I could concentrate on making the 6502 assembly code to read the input MIDI then drive the VIC-I chip. I got the prototype from eslapion, had the core MIDI read code done, but got side tracked by the VIC-I emu in chipsounds and figured I would pursue the project a bit later... that was two years ago.

Early this year Leif Bloomquist wanted to do a similar project, so I just lend him Francois's proto and my preliminary MIDI reading code and hes done VERY WELL with it See and hear it HERE

This is really a prototype, but I believe it is going to transform itself into something that any 8bit musician will not want to miss!

Congrats to Leif for pulling it off! I want serial #00001 ok?

New screenshots on official site

Its getting VERY VERY close to shipping now.
I just build release candidate 1 with the first bunch of presets.

New screen shots and a new "preset-show-off" excuse for a track from myself on

the official chipsounds page

Kudos to my testers

Hi

We are quite lucky to have some of you testing this thing.
You know who you are and you will be credited :)

Thanks again!

chipsounds on TV

Just prepping to relax and to watch a tv show whose music (by Éric Patenaude of Plogue), contains quite a few blips from your favorite in-dev synth :)

Listen to the intro and spot the blips!

LINK

Et Félicitations encore à toi Éric!

The kind of screenshot I fancy

I rechecked my DMG waveform synthesis yesterday, and made sure I had the range and AC Coupling emulation right according to the lowest 32Hz freq of the DMG Wave Channel.
(i dont have a pro mod DMG, I personnaly like the sound of the phones out of mine)

EDIT: For fairness to the scientific community, here are the actual samples.
1)chipsounds
2)DMG phone out

Slight tonal difference, but as herr_prof mentionned it can sound different from DMG to DMG.

More: Spectrum of DMG:


More: Spectrum of chipsounds:


You can save the files and animate them, as you will see the real DMG has some broadband noise at -90db, while chipsounds is pretty flat. Should i add extra noise samples to mix in? I seriously thought about adding them as extras

No fancy research, just preset making

But presets can take a while, there's just so much messing around associated with the task.

This is one of my favorite SID style arps

Testing the Wave Sequencer module

Paul Slocum's amazing ATARI synthcart beats remixed in chipsounds...

(just random pitchbend/mod wheel action on the first loop)

uPD1771C Tester App

Hum and I thought my POKEY sounded abrasive...



Until This

Making this app (in a non-existing assembler) was such a pain, it was better be worth it. This is "one sound". Most definitely some AND/OR'ing of two bit patterns. Research goes on... yummy!

ROM Trojaning the Super Cassette Vision

Hi.

Each time I do something geeky I tell myself "now THIS is THE most geeky thing I did in my life". But whatever happens, this is a phrase I tell myself month after month working on chipsounds....

Which brings us to the wonderful topic of ROM Trojaning!
But nah, lets put things in context first. :)

The idea behind chipsounds is not to put in VSTi form what people have been doing with gameboys for nearly a decade now with LSDJ/NL. The idea is to analyze, salvage and reproduce the entire crunchy 80's aural history from oblivion.

Early 80s sound chips are unique because they were EXERCISES. Nobody (except for Yannes) had a clue of what they were doing. The engineers that were given the task to come up with a soundchip were seldom musicians. So they tried and invented as they went along. Some things worked, others were dogey, but we fell in love with them anyway. Some chips were hugely popular, others faded, in most cases it was more a question of the success of their console and games more than their sound quality.

The Epoch SCV was crushed by the famicom a mere year after its launch... However, reading about  its specs, I was instantly curious about its obscure sound generator. Some sound bytes taken from the net, including this one made one thing clear, this thing didn't just generate square waves!

So a month ago I bought a PAL version of this console (renamed YENO in France), with three ROMS which I started playing with and recording right away (to get a rough idea of the spectrum of sounds it could create).
Here is some of that.

Safe to say It definitely warranted its place in chipsounds :)

But now that's where the fun parts starts. Nearly nothing is known about its soundchip, the NEC uPD1771C. Here is a picture of mine:



The only known research comes from three talented Japanese people: Mr Enri, Mr 333 and Takeda Toshiya.

Their research provided the world with a nice emulator for the console, and tons of reverse engineering details.... including a disassembler for its obscure NEC cpu.

However, the BIOS Rom for that machine (4096 Bytes) is stored inside the main cpu (NEC D7801G or uPD7801G). And its nowhere to be found online. This BIOS code holds the key to how the console talks to the chip, and without it I would have lost many (more) days to try to extract all the raw data i need for analysis. (perhaps i would have abandoned the idea altogether) To be fair, no one can blame them for not providing the bios rom online. It is indeed is a breach of copyright law! (you wont get it from me directly either)

The Mr Henri proposed a solution to grab the rom which requires a battery backed ram cartridge and some "thing" to read it back which I have no clue is what (especially with babelengrish). However I have a fascination with the trojaning research some people, including GURU have done for the MAME/MESS projects...

So I came up with a VERY LAME solution, but one that didn't require modding my rare console, or buy yet more special equipment. I however really need to give Compute's Gazette magazine some credit for my idea (MLX) ... hehe

The idea is to transform the 4096 rom bytes in something that can be printed and humanly retyped without errors, using a checksum after each line typed.

SO

1)I wrote a ROM for the console which spewed out the ENTIRE thing onto 30 PAL frames with a running check sum on each line. (You have to realize that there is NO DIRECT ASSEMBLER FOR A D7801G; only for D7810s which only as about 75% of the same byte code. Even a simple instructions like RET is different)

2)Placed the ROM on a W27C512 chip then onto a 16Kb Cart PCB:
(an electrically erasable eprom is handy when you do a hundred trials)



3)Recorded my hacked SCART/RBG/PAL output to a NTSC Canon MiniDV camera whose composite input -somehow- seem to accept its signal)

4)Transferred to the computer using Firewire

5)Saved each 30 frames separately. Here is the first one:



(note I really wanted full HEX code, but I had to be lazy somewhere right? .. so value=input-'0';

6)Batch-Massaged the picture a bit. (contrast mostly)

7)Tried to OCR the bloody thing for fun (I knew it wouldn't do any good).

8)Last resort... Back at work Monday morning, I used the whole company (we are 5 now) for a one hour intelligent typing challenge!

9)We got 'a' BIOS ROM:
MD5 : 635a978fd40db9a18ee44eff449fc126
SHA1 : 6e89d1227581c76441a53d605f9e324185f1da33
CRC32 : 7ac06182

Tried it in the eSCV emu... and it worked!!!!

I could NOW start the real work

I disassembled the BIOS and looked for the simple code that the console plays when pressing the pause button... this gave me exactly what I needed to know in order to generate the lists of values to be sent to the real chip in loop, (to check for waveform pattern changes, pitch limits etc).
The D1771C on that end is completely different from the typical !WR/!CS address/d0-d7 thing...
And its more like MIDI, in a sense that its a state machine of bytes. 4 bytes (with interrupt based ACK) to play a tone, and 10 bytes for a noise message. A pain.

Im still capturing waveforms and analyzing them, but I figured there was enough for a long overdue post on this blog.

Thanks for the Plogue Team. (Seb/Max had a tie... Seb finished first but had one checksum error. While Max finished last but had no errors)... I was pretty pathetic myself... Finished second with 7 errors... cant be good at everything :)

Here is the rom to CAPTURE the BIOS, (including the ASM file to modify it).

EDIT: Ive updated the zip file with more comments and example decoding code for anyone else attempting to dump the BIOS (including the NTSC one)

A new format to log them all.

While doing my Chip research I stumbled upon many different formats to store “chip music information”.

Some of them contain the full assembly code used by the original console/video game
-SID (C64 6510 assembly)
-NSF (NES)
-SAP (Atari 8bit computers with POKEY chips in them)
(etc)

Others only contained the actual register values WRITES along with their timestamps:
-VGM (SN, YM2612, YM2151 etc)

Both have pros and cons, But I do prefer the VGM approach, since it abstracts the chip from its processor, it allows much easier re-purposing of the music data streams. For instance, in order to use a direct PIC interface, parallel port, arduino, what-have-you to play the data back on a real chip.

For my continuing analysis and cross checking, I need to do bunch of tests on chip register data (especially for SID and POKEY).
I also need to edit entries, filter some registers, add comments to specific writes, play the result, record, analyze in a sound editor, using various home made tools.

I usually also had to type or procedurally generate long lists of chip register commands in order to hear how some chips behave under specific scenarios, and also to record a long series of chromatic notes for a sampler mapping (SID combined waveforms in chipsounds using SFZ mapping) etc, without having to delve into this or that system's native assembly each time. (I had to do that too many times that I dont miss it much)

I thought of using the VGM format and extending it but:

1)Its binary only
2)Only logs writes at 44100 Hz, (while some emulators run at full master clock of 3.57MHz or more, so you DO lose information)
3)Its been patched a lot through revisions, and is hard to maintain.
4)NIH :)

So really I just went with the simplest thing I could come up with.
VGMX!! (Extended VGM or VGM XML) .. how original

The format is very simple it doesn't require a DTD (especially since most of us use modified TinyXML source trees right?)
It goes something like this:




Note: clock and rate are two different things. Clock is the actual oscillator frequency feeding the chip, while rate is the time base for the writes. (emulator timebase in many case)

This is very preliminary, but I hope that this interest somebody.

In closing here are a few examples of real files and rendered results from my protoboard+pic setup

1)7800 Ballblazer (VGMX) (MP3)

2)C64 Great Gianna Sisters Doom (VGMX) (MP3)