openFrameworks 0.9.0: ofThreadChannel, UTF-8 and ofSoundBuffer
OF 0.9.0 Overview Part 1 - Major Changes
OF 0.9.0 Overview Part 2 - OpenGL 4.5
OF 0.9.0 Overview Part 3 - Multi-window and ofMainLoop
OF 0.9.0 Overview Part 4 - ofThreadChannel and UTF-8 support
ofThreadChannel
Multi-threaded applications are more and more common given that most computers and devices have more than one CPU.
Threading is useful if we have a process-intensive task that we don’t want to interfere with the main thread’s task which is usually drawing at 60fps. Threading provides a way to continue drawing quickly and allowing process-intensive tasks to continue in the background. Once a background task is complete, the data or result can be accessed from the main thread.
The traditional way of communicating threads in OF has been through some kind of lock before accessing shared memory between the main thread and a thread that does some kind of auxiliary work. This can be error prone since it is really easy to forget locking in the correct places and cause application crashes or lock more than necessary and cause deadlocks that freeze the application.
OF 0.9.0 comes with a new class, ofThreadChannel, that makes inter thread communication much easier without having to explicitly use mutexes or similar. You can think of a channel as a way to send information from one thread to another. Instead of having 2 threads that try to access to the same data now we have threads that access their own data and send it as a message to each other when they are done processing the data or when there’s new data to process.
Channels are thread safe so all this happens without us having to use any kind of lock in our code. Additionally the worker thread can sleep until there’s new data to be processed by calling ofThreadChannel::receive which will use 0% CPU while there’s no data to process and only wake up when there’s something to do.
There’s some examples showing how to use ofThreadChannel in examples/utils/threadChannelExample which shows some generic usage of this new class or examples/gl/threadedPixelBufferExample which shows how to download pixels from the GPU using a buffer object in a different thread. ofxThreadedImageLoader’s code is also a good example of how to use ofThreadChannel to send jobs to a worker thread and get the result back in the main thread. In all this examples there’s no explicit locking any more which makes multi threading programming way easier and less error prone.
The idea of a channel is inspired by similar utilities in some modern languages like
- Rust: https://doc.rust-lang.org/std/sync/mpsc/
- Go: http://guzalexander.com/2013/12/06/golang-channels-tutorial.html
- Clojure: http://clojure.com/blog/2013/06/28/clojure-core-async-channels.html
UTF-8 Support
As part of the removal of Poco we’ve introduced a minimal, header only library, utf8cpp. utf8cpp allows to iterate a string containing UTF-8. In 0.9.0 there’s a new class: ofUTF8Iterator which uses this library to provide an easy way to iterate over every utf-8 codepoint in a glyph.
UTF-8 is a variable length encoding which means that in a UTF-8 string we can’t do something like:
for(auto & c: str){
//do something with c
}
or
for(int i=0; i<str.size(); i++){
char c = str[i];
//do something with c
}
and expect c to be a correct UTF-8 codepoint since every character can be 1 byte, 2 bytes and even 4 bytes long depending to the alphabet it belongs.
ofUTF8Iterator wraps a string and allows to retrieve the correct codepoints from it like:
for(auto & c: ofUTF8Iterator(str)){
//do something with c
}
This is already used by OF internally to do some string manipulations like ofToLower and in the next release will allow to add support for full UTF-8 in ofTruetypefont which already lives as a pull request waiting to be merged after 0.9.0 is released: openframeworks/openFrameworks#3992
Another useful UTF-8 function in OF 0.9.0 that complements ofUTF8Iterator is ofAppendUTF8, which allows to append a UTF8 codepoint to an std::string. For example if you get a character from the keyboard callback other than the limited latin 1 range you can’t add it to a string by appending it as a character, ofAppendUTF8 allows to append any character from any alphabet by doing:
ofAppendUTF8(str,key)
ofSoundBuffer
ofSoundBuffer is part of a refactoring of the sound system that allows to simplify the callback for sound streams by passing an object instead of several parameters. It provides some basic operations over an audio buffer which make it easier to work with audio. It’s supposed to be the basis for other utils and addons which work with audio in OF. You can think of ofSoundBuffer as the equivalent to ofPixels for working with audio.