Scroll down for full-resolution images
For a while I have thought that the patterns AI makes while training are actually quite pretty to look at. Recently I’ve been hand-coding some simple AI algorithms , so I put together some visualisations that show the training progress on different datasets. Feel free to toggle on and off the different overlays on any of the images below.
Each data source has its own citations, but for a brief overall explanation of the process and what the lines on the chart actually mean, please see the section at the bottom of this post.
Yearly “Gini” US wealth gap measurement from The World Bank
The World Bank shares country-by-country data based on the Gini index. The higher the number, the more unequally wealth is distributed (where 1 is a single person having absolutely everything).
Projected life expectancy for newborns
UN shares global health and life expectancy data, split by country or grouped together globally. One number they track is “life expectancy for newborns” each year, based on national and global conditions.
Yearly carbon emissions from Our World In Data
Our World In Data has a historic CO2 and Greenhouse Gas Emissions dataset with data split by country (or grouped worldwide). This chart is based on the yearly world CO2 measurement in their dataset.
How these visualisations were created
It’s important to bear in mind that the focus here is much more on the training process than the specific data sets. The patterns of lines aren’t communicating anything specific about the original data they are just showing the steps the AI went through to try to match the patterns in that original data. In visualisations like the Gini index image – the final “best fit” line is in a different, gold colour. But in the carbon emissions image I haven’t added a different coloured line, just because it’s pretty clear what the final pattern it’s fitting is, and I think it looks better.
As some context for anyone not familiar with how AI and Machine Learning work – the most common process in an AI algorithm looks like this;
- Get some data
- Try to predict something based on that data (anything from the colour of a pixel, to the value of carbon emissions in a given year)
- Measure how bad your prediction was, and use that to automatically adjust your algorithm
- Repeat 2 and 3 until your answer is good enough to stop, or until you’ve spent so long trying you just give up.
So all I’ve done here is write a simple AI model that goes through that process for each of the data sets, and while it was going I got it to record each of the intermediate guesses it made so I could plot them on a chart. Then I saved those charts, put them into Affinity Desginer, overlayed some images and here we are.
So, the fact that there are big fans of lines at either end of the chart doesn’t communicate any extra information, they just show the steps in the training process, and look kind of pretty.