A Simple yet Powerful Artificial Intelligence Algorithm.

One of the Key Word about Interfaces today is Responsive Design: For a Mobile App, interfaces not only have to be the simplest possible, they have to be clear, and they have to propose only awaited functionalities that users will understand: they had to be adaptative.

So not surprisingly, Math Touch Book displays nothing else than what you wrote on the screen. Then touching a Math expression unfolds a Menu that proposes a short list of features according to that expression.

Capture d’écran 2016-02-02 à 11.35.32.png

Adaptative Menus are not new, but here, the program must “think” to guess the list of functionalities, operations, modifiers: Expressions can be developed or not, Simplified or not, resolved or not. And notice, some of those features can take multiple passes, like Equation resolutions.
It might surprise you, but a bit of Artificial Intelligence (AI) intervenes at this level: The same basic AI algorithm is used, both to select the menu entries and to resolve equations.

With the advent of multi-threaded mobile CPU and GPUs this last decade, Robotics and Artificial intelligence are subject to permanent breakthrough. On the software side the AI subject becomes bigger every day. But large industrial AI programs for robots, that have to compute an impressive sum of data, and more little algorithms managing equations (like  MathTouch Book does) have things in common: they both manage their objects using Graph Theory, and they both create and test algorithms before applying them for real.


Let’s have a look at what Math Touch Book does when it resolves an equation:
First, Equations are defined by Graph Trees, where the root element is an equal sign.
Elements in a tree are all attached to another. That’s what Graph Theory is about and we, computer scientists, usually think it’s a nice way to manage data.

A good Graph theory framework should also have methods for cloning graphs, detaching branches of the tree, re-attach them elsewhere, and the ability to use one or another element allocators, amongst other things.


Then we have some bricks here, that represents schematically classes available in the program, it’s a “Object Oriented” concept. This scheme tells us each of these classes are “GraphActions”, and each is able to modify a graph in their own way. They are available to be used by the Artificial Intelligence.

So now that we know the shape of the data, the tools to modify it, we want the program to find itself a way to resolve the equation. How ?

An equation resolution algorithm can be expressed as a list of “GraphActions”. But what GraphActions to use, in what order ?
And what happens if the program find a wrong algorithm ? the graph would be destroyed !
No, because we can clone the graph and test modifications on the clone.
we can also simply look at the shape of a graph, and decide to modify it in a way or another according to its shape. (You now understand the usefulness of having clone functions and special allocators in your graph framework.)
Written in pseudo-code, you can find a resolution algorithm like this:

Capture d’écran 2016-02-01 à 23.33.06

Acting innocently, with just one accumulator list, a loop and an object oriented set of classes, we have here an algorithm that … creates an algorithm. and it works !
The human brain actually does more or less this “imagine what is the most interesting thing to do, and then do it” game. The temporary allocated memory can be seen as a short-lived “imagination”.  Obviously the real code is a little bigger, but the main idea is here.


To me, the greatest of all things is that, at this level, the algorithm is found, it is available for use, but you can or cannot use it inside the application; it’s up to the user, he may use another modifier, and in some way this “search what to do” Artificial Intelligence algorithm can also be seen as a recursive extension of the Natural Intelligence of the user, that decides, the same way, to use or not a feature. So it may demonstrates that Artificial intelligence and Natural Intelligence can cooperate.
In this perspective, one already can speak of Human Enhacement.





Rendering Nice Characters In a 3D Engine

Let me tell you a developer story:Capture d’écran 2016-01-30 à 15.43.02

I always thought most vectorial font engines where limited when it comes to dynamise things a bit. I mean, drawing texts in applications: Everything is always static and monochrome.
With a Graphical API like OpenGL, and even more so, OpenGL ES for mobile, there is little hope to find a really interesting library that fits my needs.

Basically, I want all characters fitting one OpenGl texture, possibly colored, I also want my text drawing routine to make the characters dance, and finally I want to be able to create my own special characters within an existing font.
So I investigated on the best way to develop such a font engine…

Surprisingly, I found a way to not spend years creating my own font format. The two key words here are SVG and batik.

Capture d’écran 2016-01-30 à 15.19.33 SVG is a format for vectorial images, it is widely used to create nice colored vector images, and nice free applications like Inkscape are available to draw such images.
Batik is a set of open-source libraries maintained by Apache to draw and manipulate SVG, written in Java.

The interesting detail is that SVG is meant to have an inner Font format, some way to define a complete set of characters, within a SVG document. it seems obsolete, because none of the SVG tool I know use this “define my own font” feature. But there is an Open-Source “ttf to SVG Font” converter within batik. Here we are…

Capture d’écran 2016-01-30 à 15.16.41

I patched the java batik ttf converter to output another kind of SVG, that splits characters into Inkscape layers: So I can redraw  and optimize an existing font using Inkscape, and add my own characters in it.

Then I did a second tool, that converts SVG characters to a format I can draw on opengl textures. At this level, SVG helps again because it only contains simple quadratic bezier curves.Capture d’écran 2016-01-30 à 15.32.33.png

 Then I can directly use that for the real time rendering !
This engine was used to draw more funky texts in my previous game, and now it allows me to add specific mathematical symbols in my app. It inits the bitmaps very quickly, and allows each equations of Math Touch book to be drawn in one OpenGl call.
It was actually some weeks of work, but it really opened a lots of possibilities and works with all minimal OpenGl Configurations.


Why did I start Math Touch Book ?

I started mathtouchbook after I did my first video game on mobile, one year ago.
I was researching new kinds of interfaces on tablets, and also sometimes I needed to do some maths on a rough book.
I slowly realized that the equations and functions you usually draw there could gain a lot if it was managed on a haptic screen:

When you do math on a paper, you often write the first shape of an expression, then you write a second modified shape, then a third,… you constantly invert, simplify, develop, factorise what you write. Consequently, you can hardly imagine how big the result will be on the page, and mathematicians spend their lives struggling with page layouts.
Fortunately, Equations can also be seen in “four dimensions”, transforming through time.
Clearly, a mobile application could manage that aspect.

I think I could be able to program an engine using graphical processing units (gpu) that would draw equations in a vectorial space, and would be able to do morphing between those shapes.

Doing my previous video game, I actually made a Glyph Engine flexible enough to do that. Then a bit of tricks into the graph theory paradigm would be enough to make it a useful living mathematical tool.

A second important aspect of such a tool would be automatic alignment. One of the key for learning mathematics is to write clear, correctly aligned expressions. Automatizing this could help newcomers. Plus, every math tools force the users to type one line expressions with a lot of parenthesis that are hardly understandable. A vectorized space could avoid that.
Another idea came while I was reading a book about the life of Richard Feynman. He told how he could see equations in colors when he was a young boy, and how it helped him to learn.  Synesthesia, something great and easy to implement.

At this point, I knew I could make a tool within a year of work, with a quite precise workflow. So the development began.