Concept 5

Concept is now 11 years old. I manage to find less and less time to write here.

A lot of new features were added, like video API (H.264 and HEVC), more async APIs, a the main feature for Concept 5.0 is TinDB, a distributed No-SQL database system, written in Concept. Also, Concept 5 will use about 30% less memory.

The main new feature of Concept 5.0 is the fact that will drop the native desktop client. HTML5 is now mature, and there is no need for Concept Client for desktop. It will continue to have Concept Client iOS and Android native versions.

I've been working to my first big-data application, a communications application, written entirely in Concept. You can even video call me from the browser, by clicking here [call Eduard]. Every line of code is written in Concept.

I've redesigned parts of the Concept Framework for better asynchronous support.

As I've said before, the new feature in Concept 5 is TinDB.

The APIs are not final (may be modified, if needed), so this is a pre-preview. The code above inserts a Student object into test_collection, and the database server performs a Wolfram Alpha query upon insert. TinDB has stored Concept procedures.

As usual I've fixed a lot of bugs (a few of them critical). In the next few weeks I will post the entire source code on github.

A redesign for GyroGears apps is in progress, in about two weeks everything should be ready. Gyro apps are incredibly stable, so it will be mostly a face lift, in order to be more HTML5-friendly.

Now I'm working almost exclusively on TinDB and Concept Client 5.0 JS.

Workers, speech recognition and Concept 4.1

Concept 4.1 brings new optimizations, starting with recursive calls (around 50% more efficient), less memory usage for class objects ( - 50%), memory pools for objects, and workers - efficient threads that can be used by the standard Concept Core (no need to use the inefficient multi-threaded core).

Speed and memory optimizations

When testing using the binary tree benchmark (found here: http://benchmarksgame.alioth.debian.org/u64/program.php?test=binarytrees&lang=yarv, and here: http://benchmarksgame.alioth.debian.org/u64/program.php?test=binarytrees&lang=php)

PHP 5.6: Memory used - 369M to 1200M - 345 seconds
Concept 4.1: 79M to 269M - 170 seconds
Ruby 2.2.1: 176M constant - 87 seconds

Ruby is faster on this benchmark only, because it excludes the Concept JIT, how ever, when the JIT is active (ex: mathematical functions and/or loops, based on the Sieve benchmark, concept is about 20 times faster than PHP).

For the fannkuch redux benchmark (foudn here: http://benchmarksgame.alioth.debian.org/u64/program.php?test=fannkuchredux&lang=yarv)

Ruby:

Pfannkuchen(11) = 51
Time: 87435 ms

Concept:
Pfannkuchen(11) = 51
Time: 40401 ms

Ruby seems faster than Concept only on recursive calls, but I'm working on that. On the fannkuchen benchmark, Concept is more than 2x faster.

On Sieve benchmark, Concept scores 8040 iterations/second, while V8 scores on the same machine about 20000, but it uses far more memory. For now, I think that Concept has the optimal balance between speed and memory usage.

Concept Workers

When dealing with multit-hreaded applications, Concept MT core is significantly slower compared to the standard core. This is because it uses a global semaphore for synchronizing threads. The best alternative were the green threads, which are cheap, fast and don't need synchronization. The main drawback is that a blocking call, will block all the green threads. The alternative is the use of a worker.

A worker is a thread that runs in a different context, and exchange data via various APIs with the main thread.

Consider the following example:

Notice that the ChildWorker constructor simply waits for data (via Worker::Pending) and notifies the result via Worker::Result. I was inspired by the javascript web workers, with one exception: the worker runs in the same application (no need for external files).

Speech recognition

Concept 4.1 brings a wrapper for the pocketsphinx library. To see it in action, check the speech recognition example here.

As usual, lots of bugs are now fixed, new APIs for SIP protocol and DTMF, and a more advanced profiler, capable of showing the memory usage,

Concept 4.0

Concept 4.0 stable is finally released. A lot has changed, I think it has the biggest code rewrite since I've started. Is the fastest Concept core, the most memory efficient until now. On Windows it uses dlmalloc as an efficient memory allocator (replacing the standard malloc). It also uses memory pools for variables. It uses shared workers for handling multiple users in the same process, reducing the memory used, and scaling almost linearly. It has an advanced memory compensation engine, for better application feel on high-error connections. It introduces lots of new APIs, from TLS sockets, SSH and wkhtmltopdf/image to loopers (functions called when the main loop is idle).

Concept Application Server 4.0 is entirely written in Concept, using only green threads (non-blocking I/O). When compared with previous C/C++ version (3.0), it uses far less CPU cycles and memory.

Besides the core optimization, the most powerful new feature is the new JS client - Concept Client 4.0 JS (in early beta for now). Now every Concept application ever written may be run directly in a HTML 5 browser. It is tested with Chrome, Firefox, Opera and Internet Explorer 11.

Even Concept IDE and Gyro may be run in browser:

You can even try some applications. For example, a simple application for converting websites to images (png) can be found here (about 100 lines of code).

Even audio codecs are supported in browser (Speex and Opus). A simple VoIP + Video application can be found here. It has about 350 lines of code.

GyroGears will be optimized next, to generate web-friendly applications (for now the applications look like classic desktop applications).