Popular Posts

Wednesday, April 4, 2012

Tumbleweed

Meh

AlephOne had been coming along swell. See videos of it here:


Especially this one, which is AlephOne controlling ThumbJam over MIDI, where the chorusing effect is literally done by MIDI note duplication rather than as a post-processing effect:


It was designed to be a clean re-design of everything I learned when doing Geo Synthesizer, Pythagoras (initial Geo code), and Mugician, with all the icky parts rethought. It was largely successful in that regard from the standpoint of cleaning up bugs and making the code much more understandable (specifically, isolating components nicely). But between worldly distractions having nothing to do with apps, the utter crash in end-user interest a few months after Geo was released, watching what others release; I need to step back and do some soul searching.

First, I released some of the important parts of AlephOne (my private project that I only shared with a few people) on github, without any licensing terms written (I tried a license with Mugician, and they don't matter unless you really want to get lawyers involved at some point.) This code is at:


The Python vDSPCompiler

It's a mash of two ideas. The first was an abortive attempt at automating the process of turning my synthesis code into SIMD instructions that would greatly speed up AlephOne's internal engine. It was the beginning of a compiler written in Python to ingest a LISP syntax language for generating vDSP instructions, to render the entire audio buffer in parallel by sample. I got most of the timbre of AlephOne written by hand when this started. Until I figure out a few parts that would let me generate the entire effects chain (reverb, chorus, etc), the DSPCompiler class isn't of practical use. But for now, trying to automate it is a giant distraction. I might get back to that later.

Fretless MIDI

The second part is VERY important. It is the MIDI code that is used in AlephOne. Fretless.* and DeMIDI.* are the function to generate and parse MIDI from input gestures. Fretless implements all of my rants about everything that needs to be fixed with MIDI. It lets you treat MIDI much like OSC and free it from being stuck to twelve tone scales and bent notes. MIDI's abstraction here is appropriate for piano-like keyboards, and hideously/disastrously wrong for everything else. MIDI's design is liquid brain damage that gets injected into every effort to create a fine expressive instrument. It forces you to choose between stupid pitch handling and stupid polyphony. Attempts to fix it with MIDI HD look like they will be incompatible, yet tied to MIDI's past - the worst of both worlds. I am bound to MIDI just because right now, it's the only reasonable IPC between synths and controllers. You can abuse it and take advantage of ambiguities in the spec to get very close to full polyphonic pitch control, at the cost of moving all complexity into the controller and being willfully incompatible with a lot of stuff. So that's what Fretless.* does. It was explicitly designed versus ThumbJam and SampleWiz and Arctic.

So if you are designing an app that wants a fully bendy instrument with extremely natural pitch control, then this code should clear up what the hard issues are when rendering to MIDI. It boils down to creating a floating point representation of MIDI note numbers, at the cost of being limited to 16 note polyphony (by forcing one note per channel and using up to 16 of them. There is no notion of notes in this API. Rounding off pitches is the job of the controller (doing it anywhere else is premature rounding that loses important information). Generally, you need to keep separate the notions of what pitch the gesture actually implies (always slightly out of tune), what pitch you want to fret/autotune to (the "chromatic" notes and scales), and what pitch is actually rendered (somewhere in between these two things). Because touch screens draw the interface underneath the surface, all of these things must be available in the controller. The synth really only needs to know what is actually rendered. MIDI gets it backwards, because in 1980, controllers were dumb devices and the synths had the brains. It doesn't actually work like that any more. If nothing else, the new mobile device paradigm is to *install* the patch into the controller/synth device and avoid talking over the network, routing the audio instead.



I added in a new notion of legato and polyphony handling, because it's absolutely necessary for string instruments. Polyphony isn't a mode for the instrument. It's a gestural phenomenon that depends on the instrument, and is created at the controller. It is similar to channels. The instrument itself has to span many channels (to cover active and releasing notes) so that bends can be completely independent. But you can put notes into polyphony groups, which will control when notes are silenced and re-enabled for solo-mode and string-behavior such as trills. But because it is not a synthesizer/controller mode, it needs to make allowances so that chording and polyphony can be done at the same time. Related is legato, or whether the note attack is re-triggered. Generally, the first note down in a polyphony group plays the attack and every other note is a continuation of the current phase. But note that legato and polyphony are separate. In a real string instrument, the decision to pick or legato a note is made on every note - it's not a mode for the instrument that gets turned on or off.

Most importantly, it implements note-ties for MIDI. This is such a fundamental concept, that had it existed in the standard, most of the other broken-ness of MIDI might never have happened. MIDI allows the definition of pitch bend to be changed, because it assumes that you are on some kind of keyboard with a pitch wheel. But pitch wheels have a top and a bottom and a center position. The position is 14 bits. The standard interpretation is plus or minus a whole tone, where it can be increased to 12 whole tones up or down. This means that you still can't do arbitrary bend sizes (let alone *independent* bends of arbitrary size). If you have note ties, then you can dispense with all of this nonsense. If you bend a note A up to the A# position, you can do a note-tie between A bend up a semitone and rename the note as A#. You can continue to do this for as many octaves as you want, at full pitch resolution. This is exactly how written music notation actually behaves as well. Standard synths that don't understand the note-tie will experience a note-retrigger as the note is bent beyond its full up or down position. ThumbJam, SampleWiz, and Arctic understand these note ties.

I think the internal MIDI engine versus these synths is just awesome, especially on the phone. But experience with Geo suggests that maybe 10% of all people downloading the instrument can figure out how to use MIDI at all, and of those, maybe 10% ever get an understanding of why channel cycling is necessary. So if I release AlephOne as is, I will get pummeled with a lot of "why doesn't this just work!?" complaints from people who can't figure out the tool (Every hardware MIDI synth is different like VCR programming, so there is no manual you can just follow. You have to know what you are doing.). It's like trying to sell circular saws to people that need them without getting sued by people who should not have bought them. I am running out of UDID slots, so I have to either just release it as it is or shelve the project; maybe until AudioBus provides better options. (And Oh It Might!)

Original rant here:


Internal Audio Engine

I am here because I got a bit stuck and distracted when it came time to move away from just doing the MIDI part to building my own internal engine to consume the MIDI. I had to do this because no existing MIDI engine really does the behavior I need 100% correctly. So I go back into the endless task of listening to minor changes with headphones on and tweaking parameters to get the internal engine code sounding good and performant. I have no idea when AlephOne will release, but I have always had a small number of highly enthusiastic users. I am looking for inspiration at this point. I had fooled around with bringing libpd back in (which Pythagoras actually used at one point) or trying CSound. I will see where it goes.

But until then, take a look at DSPCompiler if you are using MIDI and have read some of my rants about MIDI. They aren't theoretical problems. I would move on to OSC if I could. But talking to other apps in the background with low latency only seems to have MIDI (or an abuse of it) as the only option right now.


9 comments:

  1. For the record: I love Geo. When my six month horror of a project at work is over with, I'll finally get some time to start practicing with it. I'm not sure I'll ever be good enough for the microtonal potential, but, I love how Geo deals with changing octaves. Thank you for such an expressive and fun instrument.

    ReplyDelete
  2. Don't get discouraged man! What you have done with Mugician and Geo synth is so amazing and forward-thinking. I think the value of your work is recognized by instrument makers/developers and musicians with an interest in the possible futures which touch-screen interfaces may afford.

    As a consumer, my perception of the app store scene is that there is a lot of hype/popularity contest and entitlement of the consumer which currently feeds the potential success of an app. Consumers want upgrades, new features, new voices, etc. and they want apps do everything and for it to be easy! This is in a part, I believe, a result of the 'democratization' of music production the media has produced. With time I believe the computer music literacy of the masses will grow, and your innovations will gain broader acceptance, particularly those related to instrument design; the key setup and auto-octaves of Geo synth allow for faster runs and hand-keyed arpeggios than any other app to date (unless you just want to play all your runs in C major/A minor).

    Geo synth is still the only ios app that allows me to play Hindustani ragas with the proper pitch nuance and, for the most part, correct pitch-bend ornaments. Likely, the microtonal possibilities of your instruments will continue to be appreciated by only a select few (lets face it, most people have there sense of harmony distorted by a lifetime of 12ET). But I think what your work does with instrument-design will gradually gain wider acceptance, they already have expressive possibilities beyond anything else out there.

    And I think what you are trying to do with MIDI is brave, brave, brave. Perhaps connecting with some Csound minds would be a good idea, I get the feeling that they are more interested in innovation than the dollar sign.

    Anyways, I love your work so much it sent me on a mini-rant! You are a pioneer sir, don't listen to the bollocks. And please, please keep up the good work!

    ReplyDelete
  3. Geo seriously shaked the iOS MIDI app world...
    It was about time.

    BTW: Even "non-iOS" MIDI users suck ass and whine daylong. It's not just iOS-related.

    Hang in there chum, I know what you're able to do. I may even not be close to an idea of what your next app will be.

    Keep posting!

    ReplyDelete
  4. Anonymous: I didn't mean it to sound disparaging about the fact that people will complain about the MIDI setup. It's just designed such that plug-and-play is only possible for keyboard-like instruments, and MIDI really forces end users to know too much about how MIDI works under the hood for other scenarios. The app store severely punishes learning curves, which poses a problem if you want to put up something that needs a steep learning curve to do what it must.

    As an example, I initially shipped Mugician with no frets/pitchCorrection and it had nothing but reverb and distortion level controls. Giving a list of what it did not do never stopped anybody from downloading it in the first place.

    Ratings are a strange beast, like taste in exotic foods. The same app can be 1 star or 5 stars, and it has little to do with how useful or how crashy an app is. It has a lot to do with getting people who won't like it to not download it in the first place, while getting the word out to the people who will like it.

    ReplyDelete
  5. Rob, I am extremely interested in everything you have done to date. I have learned more about music in reading your posts about the design of your apps than I ever have in my many years of making music.

    As for the interest in Geo dropping off, I think it is simple why. Those of us that love it and already have it, are busy making music with it.

    Those that have not heard of it yet, or were like me with Mugician and didn't "get it" at first, need something to trigger a look at it.

    For other apps, a weekend sale, or an update, quite often is the thing that makes me want buy it, or if I have it, check it out again. Geo hasn't had a single event of this kind, so therefore has fallen off of the radar screen of people that use app shopper apps.

    I sent some feedback to the Wizdom email address on possible enhancements, and I don't recall getting anytihng back.

    The one I would be most intersted in wold be an enhancement to octave rounding that makes it a blend of octave rounding and string polyphony. I usually have the Geo screen so that a row has as many notes as possible (the entire octave left to right), and can get some incredible runs and arpeggios that way. What I would like to see is that when you leave that "string" and go up or down, it works like when octave rounding is turned off. That way you can have a run using rounding but then jump up or down an interval that would otherwise trigger rounding. Hopefully that description makes sense.

    The other one is to be able to save globally some settings such as the style of rows/columns you prefer (see previous paragraph).

    The thing you have to remember is Geo is not an app. It is a musical instrument. It might get more notice if it was easier to record from so people could post their creations. I realize that is not the intent of it, but I create all sorts of interesting stuff that is heard only once.

    I am very interested in seeing what you come up with next. I'd love to check out AlephOne. If you need someone with Python experience and have things you need done, let me know.

    I have to add that you have also opened my eyes to other things. I thought TumbJam was a toy. When I saw you start using with AlephOne, I bought it, and I was blown away by it. There is way more to it than I had every imagined. I think the only album I have purchased on iTunes is a Dream theater album to support Jordan Rudess' support of your work. I was familiar with him and Dream Theater, but have a much greater appreciation of them now.

    The fact that you open source some of your work makes it clear to me that you are in it for the love of music and the technology of making it, not to get rich selling to the masses. I am a big open source fan having used Linux since the early days. My iPad was a 50th birthday gift that was wished for purely for making music after hearing Garageband had been released for iPad. Garageband is what it is, and there are lots of other really cool music apps too, but Geo stands above as the only true musical instrument. More than that, it is truly revolutionary. It is the only app I have come across so far that I could envision myself mastering to the point of using as a performance instrument.

    I love the fact you are thinking of open sourcing AlephOne, and did open source Mugician, but the thing I am most disturbed by is your comment on lack of interest in Geo. I had hoped you would make enough off of Geo to quit your day job and be able to make a living off of what is clearly your passion.

    I will follow and support you in whatever you choose to do.

    ReplyDelete
    Replies
    1. im glad to have such a supportive user base! my day job rocks like you would not believe, and that is part of what is going on. i have a bias for only putting out updates to fix real bugs, and new instruments to push fundamental changes. the lack of interest i was referring to was about the very small amount of user traffic i get on anything these days... mugician/geo/alephone. i have had bug fixes for geo ready to go for a very long time, but spent all time on new functionality in alephone. geo is at it limit of complexity. it wont take much for some seemingly simple changes to require removing functionality to make room for things to add. with what is currently going on with csound, libpd, possible opencl, audiobus, etc... everything is up in the air for the better. i want to simplify alephone further, so i am monitoring the situation. in the mean time, maybe some people will do something with this midi code from alephone, so my synth will have some great synths to talk to, and to offload complexity into.

      Delete
    2. oh, and Kevin did ask me about separating layout and sound presets. i think that is how it should have been from the beginning. workaround is to set layout same for all patches, and duplicate them for layouts in some cases. alephone was written with a pile of feature requests as inspiration, and i really want to avoid big changes to geo in spite of it. (nothing wrong with having multiple apps, and its best to not have a functioning app die of feature creep). geo was a rewrite of mugician with a pile of feature requests as well.

      Delete
  6. I agree with Bob regarding having the option to have octave leaping only effect a section or melodic run, with a stable 'area' for chording. Not sure exactly how it would be implemented, but I agree that is something that would be useful.

    The other feature I have wanted was better scale definition for microtonal scales, but from the youtube videos it appears that alephone has taken that into consideration.

    If you were to consider releasing an update for Geo one thing I would consider is adding some new voices and making sure that aspect gets hyped. Look at all the buzz Animoog got just for releasing some April Fool's Day patches. Perhaps a possible marketing scheme might be releasing some bug-fixes and new voices for Geo, and mentioning that the new voices are ones that will be included in an upcoming alephone release. That way you get double mileage! Of course, that would all have to be timed right. I am new to coding, so I don't know how this works from the technical perspective.. but is it just more data? Seems like some apps have almost too many voices...

    Anyways, I am not sure if you want to spend your time developing new voices, but that is yet another feature I love about Geo is the richness of the voices. I was just jamming out last night with the cathedral synth voice and iTablaPro and Bassline in the background and was like... what funky alien planet have I been transported to! Ok, so it was like 3am, but point is it was cathartic musical moment.

    Again, thank you for your efforts!

    ReplyDelete
  7. I am saddened that you feel like your work isn't being appreciated; I rave about Mugician and Geo to everyone I see using music apps on their iPads. No one else is out there pushing the boundaries like you are, and the results are really interesting and useful.

    ReplyDelete