Where do AI generated images come from?
Reversediffusion.xyz visualizes how AI-generated images emerge from a complex mathematical space built on millions of human-created training images. By zooming in on this space as a context for everything the model produces, it provides a reversed perspective on generative AI.
Where do AI generated images come from?
It’s widely known that generative AI relies on training data scraped from the public web, but the connection between this input and the final output often remains elusive. When you ask the model to create a picture of a landscape, the image you get isn’t copied from any single training image. Instead, it emerges from the complex mathematical space also known as latent space that lies between all the landscape images and their descriptions the model has learned from. Although the output can’t be traced back to one exact source, it can be positioned somewhere within this space of training data to understand the relations between the model's input and output.
Reversediffusion.xyz visualizes the latent space of the open-source Stable Diffusion model by probing it with the very training data used to build it. Like many other models, Stable Diffusion was trained on the LAION dataset—a massive collection of billions of images scraped from the web.[1] While the full dataset is enormous, many models focus on a curated subset called LAION-5B Aesthetic. This subset includes only images that another human-trained AI model has rated as having high aesthetic quality[2]. Though aesthetics are subjective, this filtering explains why generative AI often produces hyperrealistic and oversaturated images.
By encoding the 625,000 images from the LAION-5B Aesthetic >6.5 dataset into Stable Diffusion’s latent space, we can see where each training image fits within the model. $Using the UMAP algorithm to reduce this complex, high-dimensional space to three dimensions creates a spatial map–a visual representation of the model’s structure that helps us contextualize its outputs.
On this page, you can generate new images with Stable Diffusion and see their position on the map. This shows where the synthetic image belongs in latent space and which training images–whose features may have influenced the output– are nearby. A generated image of a landscape might cluster with training images of landscapes, or it might group with images sharing a similar visual style, like oil paintings or black-and-white photos. Because image generation begins with a random seed, each landscape you create will be different and appear in different locations on the map.
Once your generated image is placed in latent space, you can explore the surrounding training images. Click “see training images” to browse a selection of nearby training data[3]. Here you’ll find the original human-created images scraped from the web that informed your AI image. Clicking on any of them takes you to their original source. This way, generating an image exposes the human-made images that it originated from. And if you decide to use them, don't forget to credit the authors :-).
1. Most images in the LAION dataset were scraped from the web without the authors’ permission.
2. This project uses LAION-Aesthetics 6.5+, which filters images rated with an aesthetic quality of 6.5 or higher on a scale from 1 to 10. No images are hosted on this website.
3. The nearby image selection combines proximity in both UMAP space and the model’s latent space.
Reversediffusion.xyz visualizes how AI-generated images emerge from a complex mathematical space built on millions of human-created training images. By zooming in on this space as a context for everything the model produces, it provides a reversed perspective on generative AI.
Where do AI generated images come from?
It’s widely known that generative AI relies on training data scraped from the public web, but the connection between this input and the final output often remains elusive. When you ask the model to create a picture of a landscape, the image you get isn’t copied from any single training image. Instead, it emerges from the complex mathematical space also known as latent space that lies between all the landscape images and their descriptions the model has learned from. Although the output can’t be traced back to one exact source, it can be positioned somewhere within this space of training data to understand the relations between the model's input and output.
Reversediffusion.xyz visualizes the latent space of the open-source Stable Diffusion model by probing it with the very training data used to build it. Like many other models, Stable Diffusion was trained on the LAION dataset—a massive collection of billions of images scraped from the web.[1] While the full dataset is enormous, many models focus on a curated subset called LAION-5B Aesthetic. This subset includes only images that another human-trained AI model has rated as having high aesthetic quality[2]. Though aesthetics are subjective, this filtering explains why generative AI often produces hyperrealistic and oversaturated images.
By encoding the 625,000 images from the LAION-5B Aesthetic >6.5 dataset into Stable Diffusion’s latent space, we can see where each training image fits within the model. $Using the UMAP algorithm to reduce this complex, high-dimensional space to three dimensions creates a spatial map–a visual representation of the model’s structure that helps us contextualize its outputs.
On this page, you can generate new images with Stable Diffusion and see their position on the map. This shows where the synthetic image belongs in latent space and which training images–whose features may have influenced the output– are nearby. A generated image of a landscape might cluster with training images of landscapes, or it might group with images sharing a similar visual style, like oil paintings or black-and-white photos. Because image generation begins with a random seed, each landscape you create will be different and appear in different locations on the map.
Once your generated image is placed in latent space, you can explore the surrounding training images. Click “see training images” to browse a selection of nearby training data[3]. Here you’ll find the original human-created images scraped from the web that informed your AI image. Clicking on any of them takes you to their original source. This way, generating an image exposes the human-made images that it originated from. And if you decide to use them, don't forget to credit the authors :-).
1. Most images in the LAION dataset were scraped from the web without the authors’ permission.
2. This project uses LAION-Aesthetics 6.5+, which filters images rated with an aesthetic quality of 6.5 or higher on a scale from 1 to 10. No images are hosted on this website.
3. The nearby image selection combines proximity in both UMAP space and the model’s latent space.
Sometimes, Machine Learning models seem like magic. They can produce new patterns, texts or images seemingly out of nowhere. Powered by massive parallel computation, these algorithms reveal hidden structures in datasets so large and complex that they look like noise to us. The technology’s ability to see what is invisible to us makes it difficult to grasp what happens inside the “black box”. Noise to Signal explores the principles behind Machine Learning algorithms to expose their inner workings. Inspired by functions like Classification, Regression, Clustering, Object Detection and Denoising, it aims for a more intuitive relation to the technology and look for beauty in the black box. $ The work consists of two parts. A large panoramic screen depicts a machine learning landscape, a window into the mind of a machine trying to find patterns in unstructured data. The back reveals the inner mechanics of the massive parallel computing behind it in more detail as it shows on three screens, the relation between sequential and parallel processing and network traffic as dynamic treemaps depicting the clusters, nodes and cores that make up the system.
4 LED screens 22.2 x 3.3m, 5.8x0.76, 3.8x0.76m, realtime graphics. Coding in collaboration with: Eusebi Jucglà. Group show "The Future is Here: Awakening between AI and Art" presented by BEDI and 798Cube.
Sometimes, Machine Learning models seem like magic. They can produce new patterns, texts or images seemingly out of nowhere. Powered by massive parallel computation, these algorithms reveal hidden structures in datasets so large and complex that they look like noise to us. The technology’s ability to see what is invisible to us makes it difficult to grasp what happens inside the “black box”. Noise to Signal explores the principles behind Machine Learning algorithms to expose their inner workings. Inspired by functions like Classification, Regression, Clustering, Object Detection and Denoising, it aims for a more intuitive relation to the technology and look for beauty in the black box. $ The work consists of two parts. A large panoramic screen depicts a machine learning landscape, a window into the mind of a machine trying to find patterns in unstructured data. The back reveals the inner mechanics of the massive parallel computing behind it in more detail as it shows on three screens, the relation between sequential and parallel processing and network traffic as dynamic treemaps depicting the clusters, nodes and cores that make up the system.
4 LED screens 22.2 x 3.3m, 5.8x0.76, 3.8x0.76m, realtime graphics. Coding in collaboration with: Eusebi Jucglà. Group show "The Future is Here: Awakening between AI and Art" presented by BEDI and 798Cube.
AIR
Electric Atmospheres explores the ethereal architecture of urban datascapes. Modern life would not be possible without the electromagnetic waves carrying our data. As important as it is, we have a limited conception of how this system actually works. We imagine information magically popping up at the right place or beaming straight from a cell tower to our phones. Reality however, is much more complex and much more interesting. Electromagnetic fields are a spatial medium inhabiting spaces like cities, buildings, and our bodies. They are three-dimensional and interact with their surroundings. As we move through a city with our cell phones, we carry an electromagnetic “aura” that moves through buildings and objects and morphs its shape in reaction to their physical and electrical properties. What if we could see this dimension? What does a city look like when you visualize the interaction between architecture and information? $We cannot perceive electromagnetic waves, and the scale and makes it difficult to measure the characteristics of the electromagnetic signals around us. However, their behaviour is well understood and given a detailed three-dimensional map and enough computing power can be simulated. In collaboration with the Barcelona Supercomputing Centre and Altair Engineering I was able to simulate the behaviour of the electromagnetic cloud from a single cell-phone in different urban environments and visualize how the electromagnetic waves are absorbed and reflected by their surroundings. The results reveal an invisible dimension of reality, a hybrid of architecture and data, as cathedrals of information.
Electric Atmospheres is supported by S+T+ARTS Air, Altair Engineering, RCR Architects, Barcelona Supercomputing Centre, In4art, HLRS and Fundación Épica
Electric Atmospheres explores the ethereal architecture of urban datascapes. Modern life would not be possible without the electromagnetic waves carrying our data. As important as it is, we have a limited conception of how this system actually works. We imagine information magically popping up at the right place or beaming straight from a cell tower to our phones. Reality however, is much more complex and much more interesting. Electromagnetic fields are a spatial medium inhabiting spaces like cities, buildings, and our bodies. They are three-dimensional and interact with their surroundings. As we move through a city with our cell phones, we carry an electromagnetic “aura” that moves through buildings and objects and morphs its shape in reaction to their physical and electrical properties. What if we could see this dimension? What does a city look like when you visualize the interaction between architecture and information? $We cannot perceive electromagnetic waves, and the scale and makes it difficult to measure the characteristics of the electromagnetic signals around us. However, their behaviour is well understood and given a detailed three-dimensional map and enough computing power can be simulated. In collaboration with the Barcelona Supercomputing Centre and Altair Engineering I was able to simulate the behaviour of the electromagnetic cloud from a single cell-phone in different urban environments and visualize how the electromagnetic waves are absorbed and reflected by their surroundings. The results reveal an invisible dimension of reality, a hybrid of architecture and data, as cathedrals of information.
Electric Atmospheres is supported by S+T+ARTS Air, Altair Engineering, RCR Architects, Barcelona Supercomputing Centre, In4art, HLRS and Fundación Épica
Hyperthread explores the intertwined histories of the Microchip and the Jacquard loom. The Jacquard Loom was not only a driver of the Industrial Revolution it also kickstarted the Information Age with the mechanical processing of information through the use of punchcards, a chain of cardboard cards punched with holes that determines which cords of the fabric should be raised with each pass of the shuttle. This proto-digital system used binary code (hole - no hole) to control the behaviour of individual threads in the loom (up or down) and allowed unskilled workers to weave complex patterns. While the binary principle (hole - no hole) still underpins modern day information technology, the threads in the loom also bear striking resemblance to the physical characteristics of the microcircuits that process digital information. Modern microchips can have up to 100 layers. Electrical current flows through channels of silicon and metal passing over and under other channels like a thread going up and down to produce a complex pattern. While the loom is made to produce patterns, in microchips they are a side effect, the result of a logical routing of channels to find the shortest path between A and B. In the early days of microchip design, these patterns were hand drawn on huge sheets of paper. Today most microchips are designed parametrically. A Digital Synthesis Flow translates code that defines the required functions of the circuit into a an efficient microchip lay-out. The result is a level of complexity way beyond human comprehension. It is also quite beautiful. Hyperthread explores the aesthetics of parametric chip design through the lens of the Jacquard Loom. It is based on public domain microchips and emulators that perform different functions $such as a cryptographic key generator, a general purpose CPU or a simple flipflop (the fundamental building blocks of a microchip). Using open-source software the chips are translated from their coded instructions into three-dimensional graphical patterns. Instead of projecting these patterns on a silicon wafer at nanometer scale, they are translated into weaving instructions at millimeter scale. The result is a series of tapestries that reveal the different chips at a human scale, replacing interwoven silicon channels with coloured yarns. While the parametric patterns are not human-readable, they all share the same scale (1µm = 4mm) which allows for a comparison between simple and more complex chips. While the most complex chip (Gaussian Noise Generator) is 159x144 cm, the most simple chip (Flipflop) is only 18x16 cm. By blowing it up 4000 times, the tapestry series explores the spatial, tactile and aesthetic typologies of the common microchip.
The project uses the OpenLane Digital Synthesis flow for pattern generation and was produced at Textiellab (NL)
The series consists of Gaussian Noise Generator 159x144 cm, AES Key Generator 131x121 cm, 8080 emulator 120 x110 cm, Multiplier 104x97 cm, i4004 63x60 cm, SPM 44x42 cm, Counter 20x14 cm, Registers 18x12cm and Flipflop 18x16cm
For inquiries please contact inquiry@richardvijgen.nl
Hyperthread explores the intertwined histories of the Microchip and the Jacquard loom. The Jacquard Loom was not only a driver of the Industrial Revolution it also kickstarted the Information Age with the mechanical processing of information through the use of punchcards, a chain of cardboard cards punched with holes that determines which cords of the fabric should be raised with each pass of the shuttle. This proto-digital system used binary code (hole - no hole) to control the behaviour of individual threads in the loom (up or down) and allowed unskilled workers to weave complex patterns. While the binary principle (hole - no hole) still underpins modern day information technology, the threads in the loom also bear striking resemblance to the physical characteristics of the microcircuits that process digital information. Modern microchips can have up to 100 layers. Electrical current flows through channels of silicon and metal passing over and under other channels like a thread going up and down to produce a complex pattern. While the loom is made to produce patterns, in microchips they are a side effect, the result of a logical routing of channels to find the shortest path between A and B. In the early days of microchip design, these patterns were hand drawn on huge sheets of paper. Today most microchips are designed parametrically. A Digital Synthesis Flow translates code that defines the required functions of the circuit into a an efficient microchip lay-out. The result is a level of complexity way beyond human comprehension. It is also quite beautiful. Hyperthread explores the aesthetics of parametric chip design through the lens of the Jacquard Loom. It is based on public domain microchips and emulators that perform different functions $such as a cryptographic key generator, a general purpose CPU or a simple flipflop (the fundamental building blocks of a microchip). Using open-source software the chips are translated from their coded instructions into three-dimensional graphical patterns. Instead of projecting these patterns on a silicon wafer at nanometer scale, they are translated into weaving instructions at millimeter scale. The result is a series of tapestries that reveal the different chips at a human scale, replacing interwoven silicon channels with coloured yarns. While the parametric patterns are not human-readable, they all share the same scale (1µm = 4mm) which allows for a comparison between simple and more complex chips. While the most complex chip (Gaussian Noise Generator) is 159x144 cm, the most simple chip (Flipflop) is only 18x16 cm. By blowing it up 4000 times, the tapestry series explores the spatial, tactile and aesthetic typologies of the common microchip.
The project uses the OpenLane Digital Synthesis flow for pattern generation and was produced at Textiellab (NL)
The series consists of Gaussian Noise Generator 159x144 cm, AES Key Generator 131x121 cm, 8080 emulator 120 x110 cm, Multiplier 104x97 cm, i4004 63x60 cm, SPM 44x42 cm, Counter 20x14 cm, Registers 18x12cm and Flipflop 18x16cm
For inquiries please contact inquiry@richardvijgen.nl
Stochastic Neighbors is a series of prints based on the Archive Team’s 2009 Geocities backup in which the visual contents of the thematic neighbourhoods are organised using a t-distributed Stochastic Neighbor Embedding algorithm. Each neigborhood contains hundreds of thousands of images that are sorted by visual similarity. Geocities was a popular online community in the late 1990’s modelled after a city in cyberspace. This spatial metaphor was used not only to organise homepages into neighbourhoods and streets, but also worked as an invitation to populate cyberspace and contribute user generated content to the nascent internet. $After being acquired by Yahoo! In 1999 it was shut down in 2009 after new metaphors of the internet had taken root. A backup of 35 million homepages is what remains of this digital Pompei. Stochastic Neighbors is a reflection on this archive using machine learning algorithms.
Edition: Athens, Heartland, Hollywood, Rainforest, Tokyo. 120 x 120 cm High Definition Aluminum Dibond Prints
For inquiries please contact inquiry@richardvijgen.nl
Stochastic Neighbors is a series of prints based on the Archive Team’s 2009 Geocities backup in which the visual contents of the thematic neighbourhoods are organised using a t-distributed Stochastic Neighbor Embedding algorithm. Each neigborhood contains hundreds of thousands of images that are sorted by visual similarity. Geocities was a popular online community in the late 1990’s modelled after a city in cyberspace. This spatial metaphor was used not only to organise homepages into neighbourhoods and streets, but also worked as an invitation to populate cyberspace and contribute user generated content to the nascent internet. $After being acquired by Yahoo! In 1999 it was shut down in 2009 after new metaphors of the internet had taken root. A backup of 35 million homepages is what remains of this digital Pompei. Stochastic Neighbors is a reflection on this archive using machine learning algorithms.
Edition: Athens, Heartland, Hollywood, Rainforest, Tokyo. 120 x 120 cm High Definition Aluminum Dibond Prints
For inquiries please contact inquiry@richardvijgen.nl
Lumen Prize Finalist
When a language model produces a sentence it presents us a statistical probability based on countless texts it has analysed. Before it is able to predict the next character in a sentence it has to cut the writings of millions of authors into fragments to analyse the sequence of characters. They are stripped of meaning and structure and repurposed as a statistical resource. While the system needs the author's work as input, the results can never be traced back. But what would happen when a language model creates new texts while leaving the original work intact? What if generative AI can be traced back to and understood in the context of the original text?
The Case For a Small Language Model is a speculative AI based on the work of Dutch composer and poet Rozalie Hirs. Her 2021 poetry book Oneindige Zin (Uitgeverij Querido, 2021) which translates to Infinite Sense or Infinite Phrase in Dutch, can be read as one never ending phrase. The installation shows the entire book printed on five 30 meter long strips of labelprinter paper that scroll in both directions. As the five lines move back and forth, a vertical reading allows for new combinations to emerge. $Meanwhile a neural network based on Andrej Karpathy’s Char-RNN analyses a digital copy of Hirs’ original text and tries to create new sentences based on her work. Initially the combinations seem random and nonsensical but as the training of the neural network (running on a low power Raspberry Pi) progresses, more interesting combinations emerge. Rather than appropriating the the authors work as mere statistical data and cutting it into fragments, the system leaves the original text intact. It's output can only be read and understood in the context of the input, as the only way to display it is to move the entire manuscript text left or right.
The Case for a Small Language Network reflects on the role of authorship in generative AI and questions the practice of reducing the written expressions of millions of authors, (mostly without their permission) into a statistical resource.
In collaboration with Rozalie Hirs and Jelle Reith, supported by The Creative Industries Fund NL and the Netherlands Foundation for Literature.
When a language model produces a sentence it presents us a statistical probability based on countless texts it has analysed. Before it is able to predict the next character in a sentence it has to cut the writings of millions of authors into fragments to analyse the sequence of characters. They are stripped of meaning and structure and repurposed as a statistical resource. While the system needs the author's work as input, the results can never be traced back. But what would happen when a language model creates new texts while leaving the original work intact? What if generative AI can be traced back to and understood in the context of the original text?
The Case For a Small Language Model is a speculative AI based on the work of Dutch composer and poet Rozalie Hirs. Her 2021 poetry book Oneindige Zin (Uitgeverij Querido, 2021) which translates to Infinite Sense or Infinite Phrase in Dutch, can be read as one never ending phrase. The installation shows the entire book printed on five 30 meter long strips of labelprinter paper that scroll in both directions. As the five lines move back and forth, a vertical reading allows for new combinations to emerge. $Meanwhile a neural network based on Andrej Karpathy’s Char-RNN analyses a digital copy of Hirs’ original text and tries to create new sentences based on her work. Initially the combinations seem random and nonsensical but as the training of the neural network (running on a low power Raspberry Pi) progresses, more interesting combinations emerge. Rather than appropriating the the authors work as mere statistical data and cutting it into fragments, the system leaves the original text intact. It's output can only be read and understood in the context of the input, as the only way to display it is to move the entire manuscript text left or right.
The Case for a Small Language Network reflects on the role of authorship in generative AI and questions the practice of reducing the written expressions of millions of authors, (mostly without their permission) into a statistical resource.
In collaboration with Rozalie Hirs and Jelle Reith, supported by The Creative Industries Fund NL and the Netherlands Foundation for Literature.
1.5 million kilometers from earth where the gravitational force from the earth and our sun are in perfect balance lives a man-made object. An instrument that measures bursts of energy ejected from the surface of our star, heading towards us. These solar winds define the space weather that surrounds our Spaceship Earth.
While solar winds can be perceived as polar lights at high latitudes, they can have global consequences. As the streams of charged particles hit electronic equipment, they can disrupt communication and navigation systems and even cause power blackouts.
The frequency and intensity of solar winds follow the sun cycle that rotates the sun’s magnetic field every 11 years. As we are entering a period of increased solar activity in the next few years, space weather may become of increased significance to all.
The DSCOVR Deep Space Climate Observatory is a spacecraft that measures solar wind and provides a 15-to-60-minute advanced warning before a storm of particles and magnetic field reaches earth.
Cosmic Wind Chime is an instrument that translates the real time data measured by the DSCOVR Observatory into a small magnetic field that sets in motion a series of metal tubes. $As the spacecraft measures an increase in solar activity, the temperature, particle density and direction are transmitted to earth and translated by the chime into a dynamic magnetic field. While the tubes are pushed and pulled by the surrounding magnets they start to swing ever stronger and touch each other as the data is translated into an acoustic performance. A ring of digits displays the raw sensor data.
Solar winds take between 15 to 60 minutes to reach earth and with a delay of about 10 minutes between the measurement on board of the spacecraft and the activation of the chime, the chime performs the signal between 5 and 50 minutes before it reaches earth.
The instrument translates real time space weather into a dynamic electromagnetic performance. By visualizing and sonifying fluctuations on a scale from seconds to hours and months the work proposes a new perspective on our collective cosmic condition, the impact of our sun and its magnetic storms on our technology and a sense of connectedness in the tradition of Spaceship Earth and the Blue Marble.
1.5 million kilometers from earth where the gravitational force from the earth and our sun are in perfect balance lives a man-made object. An instrument that measures bursts of energy ejected from the surface of our star, heading towards us. These solar winds define the space weather that surrounds our Spaceship Earth.
While solar winds can be perceived as polar lights at high latitudes, they can have global consequences. As the streams of charged particles hit electronic equipment, they can disrupt communication and navigation systems and even cause power blackouts.
The frequency and intensity of solar winds follow the sun cycle that rotates the sun’s magnetic field every 11 years. As we are entering a period of increased solar activity in the next few years, space weather may become of increased significance to all.
The DSCOVR Deep Space Climate Observatory is a spacecraft that measures solar wind and provides a 15-to-60-minute advanced warning before a storm of particles and magnetic field reaches earth.
Cosmic Wind Chime is an instrument that translates the real time data measured by the DSCOVR Observatory into a small magnetic field that sets in motion a series of metal tubes. $As the spacecraft measures an increase in solar activity, the temperature, particle density and direction are transmitted to earth and translated by the chime into a dynamic magnetic field. While the tubes are pushed and pulled by the surrounding magnets they start to swing ever stronger and touch each other as the data is translated into an acoustic performance. A ring of digits displays the raw sensor data.
Solar winds take between 15 to 60 minutes to reach earth and with a delay of about 10 minutes between the measurement on board of the spacecraft and the activation of the chime, the chime performs the signal between 5 and 50 minutes before it reaches earth.
The instrument translates real time space weather into a dynamic electromagnetic performance. By visualizing and sonifying fluctuations on a scale from seconds to hours and months the work proposes a new perspective on our collective cosmic condition, the impact of our sun and its magnetic storms on our technology and a sense of connectedness in the tradition of Spaceship Earth and the Blue Marble.
As you descend the stairs into the central hall of the Delft train station, a panoramic screen shows a subterranean view of Delft. It portrays a subset of around 30.000 sample analyses from Terraindex’s enormous collection of soil data. The result is a view of Delft from below where each line represents a soil sample that matches the viewers perspective. Moving through the data landscape you encounter clusters of soil findings. $Some are natural such as sand or shells, others are man made such as asphalt, plastic or brick. The visualisation cycles trough a series of routes that connect these clusters. By following these routes you are presented with an ever changing View Beneath Delft.
In collaboration with TerraIndex
As you descend the stairs into the central hall of the Delft train station, a panoramic screen shows a subterranean view of Delft. It portrays a subset of around 30.000 sample analyses from Terraindex’s enormous collection of soil data. The result is a view of Delft from below where each line represents a soil sample that matches the viewers perspective. Moving through the data landscape you encounter clusters of soil findings. $Some are natural such as sand or shells, others are man made such as asphalt, plastic or brick. The visualisation cycles trough a series of routes that connect these clusters. By following these routes you are presented with an ever changing View Beneath Delft.
In collaboration with TerraIndex
Bosch Centraal is a speculative proposal for a train station that is both a city park and a gravity battery. Since 2017 the Dutch railroads use renewable electricity from wind parks off te coast of the Netherlands, Sweden and Finland. While this is a good step in the energy transition it is also a theoretical reality. There is no direct relation between the production of electricity and the train. If there is not enough wind during the day, the train gets its power from a local gas plant while if it’s windy at night the electricity might get lost because there is no demand. Bosch Centraal aims to restore the relation between power production and consumption by lifting the roof of the station when electricity is abundantly available during the night and release electricity by lowering it during the day. $As such it balances the power grid but also creates a readable and fluctuating architecture. As the roof is raised and lowered it creates an ever changing spacial experience of the station while the crane towers function as gauging rods that show the amount of power stored in the raised roof. The towers also function as variable elevators that give access to a rooftop park that provides both the weight for the gravity battery as well as an ever changing view of the city. Instead of hiding energy storage in an industrial box outside the city, Bosch Centraal proposes urban energy storage that celebrates the energy transition as an opportunity for new types of public spaces.
Bosch Centraal is a speculative proposal for a train station that is both a city park and a gravity battery. Since 2017 the Dutch railroads use renewable electricity from wind parks off te coast of the Netherlands, Sweden and Finland. While this is a good step in the energy transition it is also a theoretical reality. There is no direct relation between the production of electricity and the train. If there is not enough wind during the day, the train gets its power from a local gas plant while if it’s windy at night the electricity might get lost because there is no demand. Bosch Centraal aims to restore the relation between power production and consumption by lifting the roof of the station when electricity is abundantly available during the night and release electricity by lowering it during the day. $As such it balances the power grid but also creates a readable and fluctuating architecture. As the roof is raised and lowered it creates an ever changing spacial experience of the station while the crane towers function as gauging rods that show the amount of power stored in the raised roof. The towers also function as variable elevators that give access to a rooftop park that provides both the weight for the gravity battery as well as an ever changing view of the city. Instead of hiding energy storage in an industrial box outside the city, Bosch Centraal proposes urban energy storage that celebrates the energy transition as an opportunity for new types of public spaces.
When a computer looks at the world, what does it “see”? This installation lets the audience look at 558 episodes of VPRO Tegenlicht (Dutch Future Affairs Documentary series) through the eyes of a computer vision Neural Network. The installation employs an object detection algorithm that has been trained on Imagenet data across three categories; people, artefacts and natural objects. These categories are the "eyes" through which the algorithm observes the archive.
By choosing an eye, the user chooses a narrow perspective; The eye trained on people can only see people. With these eyes, the algorithm can identify between 30 and 60 different classes; men, women, children, but also politicians, economists and "bad people". These classes follow the structure of the imagenet dataset and range from obvious and utilitarian to absurd and problematic. As such they highlight the inevitable cultural perspectives embedded in the process of computer vision; from data collection to annotation and categorization. After choosing an eye and browsing through its different classes the user can set the confidence threshold, a value that sets how "confident" the algorithm needs to be for a result to appear. By playing with this value, the audience can explore the limits of the algorithm; where does it draw the line between a man and a woman? By exposing these choices; the eye, the class and the confidence, the installation allows for a more intuitive relation to A.I.$One that does not present A.I. as an outcome or an answer, but as a dialog between man and machine. Sometimes powerful and revealing, but also fallible and coloured.
After training the algorithm it has observed over 400 hours of video and detected more than 1 million objects. These objects are stored in a database as short clips of bounding boxes of ever changing scale and position. The installation presents these objects both in a linear way, where the bounding boxes are shown within the frame of the film as well as a non linear way, where the objects exist as autonomous entities, isolated from the original frame.
From February 20th to June 26th 2022 at Het Nieuwe Instituut Rotterdam
Installation Design and Production: Richard Vijgen | Curatorial Team: Het Nieuwe Instituut, VPRO Tegenlicht (Bregtje Van Der Haak), Richard Vijgen, Koehorst in ’t Veld | Exhibition Design: Koehorst in 't Veld | Programme Manager: Olle Lundin | Production: Floor Berkhout, Babette Zijlstra | Sound Design: Eusebi Jucglà | Supported by Het Nieuwe Instituut, VPRO, Mondriaan Fonds, Nederlands Instituut voor Beeld en Geluid
When a computer looks at the world, what does it “see”? This installation lets the audience look at 558 episodes of VPRO Tegenlicht (Dutch Future Affairs Documentary series) through the eyes of a computer vision Neural Network. The installation employs an object detection algorithm that has been trained on Imagenet data across three categories; people, artefacts and natural objects. These categories are the "eyes" through which the algorithm observes the archive.
By choosing an eye, the user chooses a narrow perspective; The eye trained on people can only see people. With these eyes, the algorithm can identify between 30 and 60 different classes; men, women, children, but also politicians, economists and "bad people". These classes follow the structure of the imagenet dataset and range from obvious and utilitarian to absurd and problematic. As such they highlight the inevitable cultural perspectives embedded in the process of computer vision; from data collection to annotation and categorization. After choosing an eye and browsing through its different classes the user can set the confidence threshold, a value that sets how "confident" the algorithm needs to be for a result to appear. By playing with this value, the audience can explore the limits of the algorithm; where does it draw the line between a man and a woman? By exposing these choices; the eye, the class and the confidence, the installation allows for a more intuitive relation to A.I.$One that does not present A.I. as an outcome or an answer, but as a dialog between man and machine. Sometimes powerful and revealing, but also fallible and coloured.
After training the algorithm it has observed over 400 hours of video and detected more than 1 million objects. These objects are stored in a database as short clips of bounding boxes of ever changing scale and position. The installation presents these objects both in a linear way, where the bounding boxes are shown within the frame of the film as well as a non linear way, where the objects exist as autonomous entities, isolated from the original frame.
From February 20th to June 26th 2022 at Het Nieuwe Instituut Rotterdam
Installation Design and Production: Richard Vijgen | Curatorial Team: Het Nieuwe Instituut, VPRO Tegenlicht (Bregtje Van Der Haak), Richard Vijgen, Koehorst in ’t Veld | Exhibition Design: Koehorst in 't Veld | Programme Manager: Olle Lundin | Production: Floor Berkhout, Babette Zijlstra | Sound Design: Eusebi Jucglà | Supported by Het Nieuwe Instituut, VPRO, Mondriaan Fonds, Nederlands Instituut voor Beeld en Geluid
On Christmas Eve 2019 the University of Maastricht (NL) was hit by a severe cyberattack. A few weeks before, an employee of the university had accidentally clicked a link in a phishing mail that linked to an excel file called schedule.xls. This file installed the SDBBot malware on the employee’s laptop. Within a few days the attacker was able to access and spread across the University’s local network. Using a so called EternalBlue exploit on an outdated server the attacker was able to get full access to the central system. On december 23rd 2019 at 18:52 the attacker had installed ransomware that encrypted all files on 267 of the University’s servers containing everything from student affairs, to payroll to researchers and phd’s project data. A ransom letter was left in a .txt file demanding € 200.000 in bitcoin.
Unable to function, the University decided to pay and take the whole process public by publishing the digital forensic report and hosting a symposium to educate the public about the course of events.While the attack has had a profound impact on the University, the staff and the students, there is little that remains of the event other than a sealed off laptop with a sticker on it that reads “Patient zero. Do not connect to the network”.
$
Eternal Blue is a monument to the events of 2019 as well as a window into an invisible dimension of reality. Each day the University’s firewall intercepts tens of thousands of malicious packets. Over the course of 24 hours these packets reveal the patterns of a global digital tide with attacks originating from different parts of the world in correlation with day and night rhythms. Each intercepted package is logged and visualised as a coloured pixel. Together they form a slowly rotating sphere that represents 24 hours from top to bottom where the color represents the country of origin. A period of continued attacks from a specific country appears as a coloured band that starts at the top slowly moving down. Every day reveals a new pattern based on the origin and intensity of the attacks.
While one day the sphere may be predominantly blue with a few bands of color, the next day it may be mostly green with a few specks of yellow and red. As such it resembles a contemporary clock that measures the passing of time by the intensity of digital attacks. It is a the invisible but very real background against which the University functions as an open and international institution.
On Christmas Eve 2019 the University of Maastricht (NL) was hit by a severe cyberattack. A few weeks before, an employee of the university had accidentally clicked a link in a phishing mail that linked to an excel file called schedule.xls. This file installed the SDBBot malware on the employee’s laptop. Within a few days the attacker was able to access and spread across the University’s local network. Using a so called EternalBlue exploit on an outdated server the attacker was able to get full access to the central system. On december 23rd 2019 at 18:52 the attacker had installed ransomware that encrypted all files on 267 of the University’s servers containing everything from student affairs, to payroll to researchers and phd’s project data. A ransom letter was left in a .txt file demanding € 200.000 in bitcoin.
Unable to function, the University decided to pay and take the whole process public by publishing the digital forensic report and hosting a symposium to educate the public about the course of events.While the attack has had a profound impact on the University, the staff and the students, there is little that remains of the event other than a sealed off laptop with a sticker on it that reads “Patient zero. Do not connect to the network”.
$
Eternal Blue is a monument to the events of 2019 as well as a window into an invisible dimension of reality. Each day the University’s firewall intercepts tens of thousands of malicious packets. Over the course of 24 hours these packets reveal the patterns of a global digital tide with attacks originating from different parts of the world in correlation with day and night rhythms. Each intercepted package is logged and visualised as a coloured pixel. Together they form a slowly rotating sphere that represents 24 hours from top to bottom where the color represents the country of origin. A period of continued attacks from a specific country appears as a coloured band that starts at the top slowly moving down. Every day reveals a new pattern based on the origin and intensity of the attacks.
While one day the sphere may be predominantly blue with a few bands of color, the next day it may be mostly green with a few specks of yellow and red. As such it resembles a contemporary clock that measures the passing of time by the intensity of digital attacks. It is a the invisible but very real background against which the University functions as an open and international institution.
Atmospheric Lighthouse is a visual interface between the city and it’s environment, a visualization of the interaction between the atmosphere and the city. As the wind blows fresh air into the streets it carries dust and sand from the desert, moisture from the sea or cool air from the mountains, linking the city with places hundreds of kilometers away. The flow or air creates an Urban Canyon Effect as it bounces off and around high buildings. It absorbs the particles created by burning fossil fuels and carries them out of the city.$This complex interaction between air flow and the built environment is largely invisible but fundamental to urban life. Atmospheric Lighthouse captures and analyses realtime atmospheric data to visualize the flow of air around the skin of Torre Glòries using a 43 x 120 pixel grid. Wind speed and direction result in ever changing patterns while measurements of CO2, small particles (PM10, PM2.5) and NOx affect the color of the air flows.
Coding: Richard Vijgen and Eusebi Jucglà
Atmospheric Lighthouse is a visual interface between the city and it’s environment, a visualization of the interaction between the atmosphere and the city. As the wind blows fresh air into the streets it carries dust and sand from the desert, moisture from the sea or cool air from the mountains, linking the city with places hundreds of kilometers away. The flow or air creates an Urban Canyon Effect as it bounces off and around high buildings. It absorbs the particles created by burning fossil fuels and carries them out of the city.$This complex interaction between air flow and the built environment is largely invisible but fundamental to urban life. Atmospheric Lighthouse captures and analyses realtime atmospheric data to visualize the flow of air around the skin of Torre Glòries using a 43 x 120 pixel grid. Wind speed and direction result in ever changing patterns while measurements of CO2, small particles (PM10, PM2.5) and NOx affect the color of the air flows.
Coding: Richard Vijgen and Eusebi Jucglà
A data-driven portrait of Barcelona that visualises the city as an interconnected flow system of human and non-human actors.
Hyperview Barcelona is one part of Mirador Torre Glories, a permanent two part exhibition that connects the basement of the landmark Torre Glories with the 144 meters high observation deck. Each part gives a different perspective on the city. While the observation deck focusses on the visible, combining a spectacular view with an immersive installation by Tomás Saraceno, the basement is dedicated to the invisible; the city as a network of people, materials, energy and biology.
Rhythms is a holographic installation that visualises a realtime perspective on the city by combining 10 different live data sources ranging from from weather stations, air quality and traffic sensors to electromagnetic signals and lidar data into a highly dynamic story narrated by an A.I. voice.
The story consists of three chapters; flow, pulse and vibrations. Chapter one focusses on the interaction between air flow and the city grid$by using realtime wind data to simulate the urban canyon effect (how air flows through a city) and its effects on trees. Chapter two uses realtime traffic data to visualise the pulse of the city through movement of people and goods. Chapter tree looks at the flow of information through the city as electromagnetic vibrations. It uses a SDR receiver to scan the electromagnetic spectrum in real time and uses cell tower data to transform a high resolution Lidar map as well as various API’s to monitor recent usage of social media in the city.
Combined with a data driven musical score by John Talabot, the result is a 7:30 minute volumetric composition that changes with the rhythm of the city.
Design and Production: Richard Vijgen
Coding: Richard Vijgen and Eusebi Jucglà
Music: John Talabot
Commissioned by: Mediapro Exhibitions
Permanent exhibtion from May 2022
A data-driven portrait of Barcelona that visualises the city as an interconnected flow system of human and non-human actors.
Hyperview Barcelona is one part of Mirador Torre Glories, a permanent two part exhibition that connects the basement of the landmark Torre Glories with the 144 meters high observation deck. Each part gives a different perspective on the city. While the observation deck focusses on the visible, combining a spectacular view with an immersive installation by Tomás Saraceno, the basement is dedicated to the invisible; the city as a network of people, materials, energy and biology.
Rhythms is a holographic installation that visualises a realtime perspective on the city by combining 10 different live data sources ranging from from weather stations, air quality and traffic sensors to electromagnetic signals and lidar data into a highly dynamic story narrated by an A.I. voice.
The story consists of three chapters; flow, pulse and vibrations. Chapter one focusses on the interaction between air flow and the city grid$by using realtime wind data to simulate the urban canyon effect (how air flows through a city) and its effects on trees. Chapter two uses realtime traffic data to visualise the pulse of the city through movement of people and goods. Chapter tree looks at the flow of information through the city as electromagnetic vibrations. It uses a SDR receiver to scan the electromagnetic spectrum in real time and uses cell tower data to transform a high resolution Lidar map as well as various API’s to monitor recent usage of social media in the city.
Combined with a data driven musical score by John Talabot, the result is a 7:30 minute volumetric composition that changes with the rhythm of the city.
Design and Production: Richard Vijgen
Coding: Richard Vijgen and Eusebi Jucglà
Music: John Talabot
Commissioned by: Mediapro Exhibitions
Permanent exhibtion from May 2022
2020
Hertzian Landscapes is a live visualization of the radio spectrum. Unlike visible light, waves in the radio spectrum cannot be perceived by us directly yet this space is teeming with human activity. Hertzian Landscapes employs a digital receiver to scan large swaths of radio spectrum in near real-time and visualizes thousands of signals into a panoramic electromagnetic landscape. Users can zoom in to specific frequencies by positioning themselves in front of the panorama as if controlling a radio tuner with their body, giving them a sense of walking through the spectrum.
From radio broadcasts to weather satellites and from medical implants to aeronautical navigation,$the radio spectrum is divided into hundreds of designated slices each tied to a specific application. Based on a localized frequency database that describes these slices, signals are annotated to provide information about their theoretical type and application.
Hertzian Landscapes is supported by the Netherlands Institute for Sound and Vision and the Netherlands Creative Industries Fund.
2020 Lumen Prize 3D and Interactive 2021 S+T+ARTS STS Award
Hertzian Landscapes is a live visualization of the radio spectrum. Unlike visible light, waves in the radio spectrum cannot be perceived by us directly yet this space is teeming with human activity. Hertzian Landscapes employs a digital receiver to scan large swaths of radio spectrum in near real-time and visualizes thousands of signals into a panoramic electromagnetic landscape. Users can zoom in to specific frequencies by positioning themselves in front of the panorama as if controlling a radio tuner with their body, giving them a sense of walking through the spectrum.
From radio broadcasts to weather satellites and from medical implants to aeronautical navigation,$the radio spectrum is divided into hundreds of designated slices each tied to a specific application. Based on a localized frequency database that describes these slices, signals are annotated to provide information about their theoretical type and application.
Hertzian Landscapes is supported by the Netherlands Institute for Sound and Vision and the Netherlands Creative Industries Fund.
2020 Lumen Prize 3D and Interactive 2021 S+T+ARTS STS Award
How Forests Think is an installation for the Oerol Cultural Festival on the Island of Terschelling.
The work proposes a reversed perspective on our natural surroundings by exploring how they perceive us. A series of geophones, an acoustic detector that responds to ground vibrations generated by seismic waves, were installed on a forest plot on the island. Each geophone was connected to a custom digital amplifier that made soil vibrations audible and emphasised a specific part of the spectrum. Each amplifier was then connected to 4 listening posts with headphones.
Due to the sensitivity of the geophones this resulted in a series of real time soundscapes that included the footsteps of people up to 50 meters away, $the sound of insects near the sensor and the vibrations of the the wind passed on from the treetops via the trunks to the roots and the soil. Waves hitting the nearby shore would add a low frequency humming.
In addition to the audio installation the work consists of a series of observation points with microscopic videos of soil samples and a circular contemplation space.
The work is part of the second instalment of a 4 year project by artist Elmo Vermijs that focusses on different stages of the forest’s life cycle.
How Forests Think is an installation for the Oerol Cultural Festival on the Island of Terschelling.
The work proposes a reversed perspective on our natural surroundings by exploring how they perceive us. A series of geophones, an acoustic detector that responds to ground vibrations generated by seismic waves, were installed on a forest plot on the island. Each geophone was connected to a custom digital amplifier that made soil vibrations audible and emphasised a specific part of the spectrum. Each amplifier was then connected to 4 listening posts with headphones.
Due to the sensitivity of the geophones this resulted in a series of real time soundscapes that included the footsteps of people up to 50 meters away, $the sound of insects near the sensor and the vibrations of the the wind passed on from the treetops via the trunks to the roots and the soil. Waves hitting the nearby shore would add a low frequency humming.
In addition to the audio installation the work consists of a series of observation points with microscopic videos of soil samples and a circular contemplation space.
The work is part of the second instalment of a 4 year project by artist Elmo Vermijs that focusses on different stages of the forest’s life cycle.
WiFi Impressionist is a field installation that draws electromagnetic landscapes inspired by the cityscapes of William Turner. The work consists of a directional antenna on a pan-tilt mechanism that listens for WiFi signals and builds a three dimensional model of the signals around it. From this model a viewport is selected that defines the perspective and the frame. Signals that are picked up within the frame are visualised as waves emitted from a specific origin and drawn using a mobile plotter.$The antenna and the plotter are both mounted on a tripod and can be placed in the field much like a painter would set up his easel. Once positioned and oriented a drawing becomes denser over time depending on the density of networks around it. Wherever there is a WiFi signal, the drawing will eventually fill the frame.
YouFab Global
2019 Finalist
WiFi Impressionist is a field installation that draws electromagnetic landscapes inspired by the cityscapes of William Turner. The work consists of a directional antenna on a pan-tilt mechanism that listens for WiFi signals and builds a three dimensional model of the signals around it. From this model a viewport is selected that defines the perspective and the frame. Signals that are picked up within the frame are visualised as waves emitted from a specific origin and drawn using a mobile plotter.$The antenna and the plotter are both mounted on a tripod and can be placed in the field much like a painter would set up his easel. Once positioned and oriented a drawing becomes denser over time depending on the density of networks around it. Wherever there is a WiFi signal, the drawing will eventually fill the frame.
YouFab Global
2019 Finalist
Throughout Palermo you can see the sky not only from the streets but also through the illusionistic ceiling paintings in the city's many palaces. These paintings typically use the perspective techniques di sotto in su (paintings seen from below) and quadratura (a highly realistic perspective that extends the architectural space). Inspired by these painting techniques, Connected by Air creates a data visualisation of the sky, di sotto in su.
The projection of Palermo’s sky on the room’s ceiling in Palazzo Ajutamicristo recreates a window that provides a comprehensive overview of all the data and objects that fill the sky. It includes wireless signals (2G, 3G, 4G coverage), satellites, air traffic (flight patterns), air conditions (particles, dust), and air flow (wind patterns).$In addition, it visualises the wireless activity caused by visitor’s devices as they try to connect to the cloud through the “opening in the ceiling”. The visualisation changes colour throughout the day to match light conditions outside which adds to the effect of Quadratura.
Connected by Air projects a contemporary sky’s image as a carrier of people, matter and information.
Developed with the support of Creative Industries Fund NL and Dutch Culture
Throughout Palermo you can see the sky not only from the streets but also through the illusionistic ceiling paintings in the city's many palaces. These paintings typically use the perspective techniques di sotto in su (paintings seen from below) and quadratura (a highly realistic perspective that extends the architectural space). Inspired by these painting techniques, Connected by Air creates a data visualisation of the sky, di sotto in su.
The projection of Palermo’s sky on the room’s ceiling in Palazzo Ajutamicristo recreates a window that provides a comprehensive overview of all the data and objects that fill the sky. It includes wireless signals (2G, 3G, 4G coverage), satellites, air traffic (flight patterns), air conditions (particles, dust), and air flow (wind patterns).$In addition, it visualises the wireless activity caused by visitor’s devices as they try to connect to the cloud through the “opening in the ceiling”. The visualisation changes colour throughout the day to match light conditions outside which adds to the effect of Quadratura.
Connected by Air projects a contemporary sky’s image as a carrier of people, matter and information.
Developed with the support of Creative Industries Fund NL and Dutch Culture
Electronica
The infosphere[1] relies on an intricate network of signals, wired and wireless, that support it. We are completely surrounded by an invisible system of data cables and radio signals from access points, cell towers and overhead satellites. Our digital lives depend on these very physical systems for communication, observation and navigation.
The Architecture of Radio is a site-specific iPad application that visualizes this network of networks by reversing the ambient nature of the infosphere; hiding the visible while revealing the invisible technological landscape we interact with through our devices.
The architecture of radio app is a realtime, location based visualization of cell towers, wifi routers, communication, navigation and observation satellites and their signals.$A site specific version of the app includes wired communication infrastructure embedded in the exhibition space. It's aim is to provide a comprehensive window into the infosphere.
[1]The infosphere refers to an interdependent environment, like a biosphere, that is populated by informational entities. While an example of the sphere of information is cyberspace, infospheres are not limited to purely online environments.
The infosphere[1] relies on an intricate network of signals, wired and wireless, that support it. We are completely surrounded by an invisible system of data cables and radio signals from access points, cell towers and overhead satellites. Our digital lives depend on these very physical systems for communication, observation and navigation.
The Architecture of Radio is a site-specific iPad application that visualizes this network of networks by reversing the ambient nature of the infosphere; hiding the visible while revealing the invisible technological landscape we interact with through our devices.
The architecture of radio app is a realtime, location based visualization of cell towers, wifi routers, communication, navigation and observation satellites and their signals.$A site specific version of the app includes wired communication infrastructure embedded in the exhibition space. It's aim is to provide a comprehensive window into the infosphere.
[1]The infosphere refers to an interdependent environment, like a biosphere, that is populated by informational entities. While an example of the sphere of information is cyberspace, infospheres are not limited to purely online environments.
Awards 2017
Digital networks are forever expanding. Places without cell phone reception or Wi-Fi connection are increasingly hard to find. The remaining White Spots on the digital map will soon disappear, leaving no place on earth unconnected. But what is happening off the grid?
White Spots is a collaborative multimedia project by information designer Richard Vijgen, documentary filmmaker Bregtje van der Haak, and visual artist Jacqueline Hassink. Working in various media, they travel beyond the frontiers of the networked world to explore unwired landscapes, communities, and lifestyles, questioning the need to be always connected in one seamless, planetary Tech-Topia. While scarcely populated areas experience low connectivity for obvious economic reasons, the journey brings surprising stories of an often deliberate lack of connectivity, even inside the world's most intensely networked digital hubs. $In VR mode, the network scanner shows the invisible digital signals around you in real time and takes you on a journey to the end of the Internet in immersive 360° stories. In Map mode, the White Spots world map shows the global divide between the connected and unconnected worlds. Browse the map to explore video stories about life off the grid or use the route planner to venture into uncharted territory yourself. The route planner finds a route to a White Spot near you and invites you to add new stories to the map.
The White Spots App features a world map, a network scanner, a GPS based route planner, short documentary clips and a series of virtual reality experiences.
Digital networks are forever expanding. Places without cell phone reception or Wi-Fi connection are increasingly hard to find. The remaining White Spots on the digital map will soon disappear, leaving no place on earth unconnected. But what is happening off the grid?
White Spots is a collaborative multimedia project by information designer Richard Vijgen, documentary filmmaker Bregtje van der Haak, and visual artist Jacqueline Hassink. Working in various media, they travel beyond the frontiers of the networked world to explore unwired landscapes, communities, and lifestyles, questioning the need to be always connected in one seamless, planetary Tech-Topia. While scarcely populated areas experience low connectivity for obvious economic reasons, the journey brings surprising stories of an often deliberate lack of connectivity, even inside the world's most intensely networked digital hubs. $In VR mode, the network scanner shows the invisible digital signals around you in real time and takes you on a journey to the end of the Internet in immersive 360° stories. In Map mode, the White Spots world map shows the global divide between the connected and unconnected worlds. Browse the map to explore video stories about life off the grid or use the route planner to venture into uncharted territory yourself. The route planner finds a route to a White Spot near you and invites you to add new stories to the map.
The White Spots App features a world map, a network scanner, a GPS based route planner, short documentary clips and a series of virtual reality experiences.
During the Dutch Design Week 2017, Het Nieuwe Instituut organised the Embassy of Data, an exhibition about the role of public and private data in the city. For this exhibition I developed a data panorama that visualizes all smart-city infrastructure in a 400 meter radius around the exhibition space in the city center of Eindhoven [NL] (Eindhoven is the leading Smart City in the Netherlands). The panorama features data from OpenStreetMap, municipal camera's, cell towers, water-level sensors, directional microphones, air quality sensors, motion, traffic and crowd detection, "City Beacons", citizen classification data and more than a 100.000 geolocated public notifications. All data is presented light sources in a 360 degree panorama in a way that lights up area's of the city that are more heavily monitored than others.$By adding the physical sensors that are used in the city to the installation, it becomes a kind of diorama, helping the audience to recognise the sensors in the city and understand their function. The installation aims to translate the hidden abstraction of these technologies into a readable experience for a broad audience, providing a sense of ownership necessary for a nuanced discussion about the future of smart cities.
In collaboration with Linda Vlassenrood
Exhibition Design: Koehorst in 't Veld
During the Dutch Design Week 2017, Het Nieuwe Instituut organised the Embassy of Data, an exhibition about the role of public and private data in the city. For this exhibition I developed a data panorama that visualizes all smart-city infrastructure in a 400 meter radius around the exhibition space in the city center of Eindhoven [NL] (Eindhoven is the leading Smart City in the Netherlands). The panorama features data from OpenStreetMap, municipal camera's, cell towers, water-level sensors, directional microphones, air quality sensors, motion, traffic and crowd detection, "City Beacons", citizen classification data and more than a 100.000 geolocated public notifications. All data is presented light sources in a 360 degree panorama in a way that lights up area's of the city that are more heavily monitored than others.$By adding the physical sensors that are used in the city to the installation, it becomes a kind of diorama, helping the audience to recognise the sensors in the city and understand their function. The installation aims to translate the hidden abstraction of these technologies into a readable experience for a broad audience, providing a sense of ownership necessary for a nuanced discussion about the future of smart cities.
In collaboration with Linda Vlassenrood
Exhibition Design: Koehorst in 't Veld
The infosphere* relies on an intricate network of signals, wired and wireless, that support it. We are completely surrounded by an invisible system of data cables and radio signals from access points, cell towers and overhead satellites. Our digital lives depend on these very physical systems for communication, observation and navigation. The Architecture of Radio is a site-specific iPad application that visualizes this network of networks by reversing the ambient nature of the infosphere; hiding the visible while revealing the invisible technological landscape we interact with through our devices.
The architecture of radio app is a realtime, location based visualization of cell towers, wifi routers, communication, navigation and observation satellites and their signals. A site specific version of the app includes wired communication infrastructure embedded in the exhibition space. It's aim is to provide a comprehensive window into the infosphere. $I created a large scale 360 degree projection of Architecture of Radio for STRP Biënnale 2017. A circular projection, 10 meters in diameter displays a wireless landscape that extends the visualisation in the iPad app. Where the app visualises all cell towers within range of the device (most within one kilometre distance), the 360º panorama visualises the wireless landcape beyond that (up to 100 km's).
Photo credit: Ruud Balk
*The infosphere refers to an interdependent environment, like a biosphere, that is populated by informational entities. While an example of the sphere of information is cyberspace, infospheres are not limited to purely online environments.
The infosphere* relies on an intricate network of signals, wired and wireless, that support it. We are completely surrounded by an invisible system of data cables and radio signals from access points, cell towers and overhead satellites. Our digital lives depend on these very physical systems for communication, observation and navigation. The Architecture of Radio is a site-specific iPad application that visualizes this network of networks by reversing the ambient nature of the infosphere; hiding the visible while revealing the invisible technological landscape we interact with through our devices.
The architecture of radio app is a realtime, location based visualization of cell towers, wifi routers, communication, navigation and observation satellites and their signals. A site specific version of the app includes wired communication infrastructure embedded in the exhibition space. It's aim is to provide a comprehensive window into the infosphere. $I created a large scale 360 degree projection of Architecture of Radio for STRP Biënnale 2017. A circular projection, 10 meters in diameter displays a wireless landscape that extends the visualisation in the iPad app. Where the app visualises all cell towers within range of the device (most within one kilometre distance), the 360º panorama visualises the wireless landcape beyond that (up to 100 km's).
Photo credit: Ruud Balk
*The infosphere refers to an interdependent environment, like a biosphere, that is populated by informational entities. While an example of the sphere of information is cyberspace, infospheres are not limited to purely online environments.
WifiTapestry is a dynamic wall hanging that visualises the wireless activity of a space. The tapestry visualises the ever changing "landscape" of radio frequencies around us. The invisible signals from Cellphones, printers and all kinds of smart devices leave an imprint as they try to negotiate available wireless channels. A controller listens to all traffic across 13 channels of the 2.4GHz WiFi Spectrum. $Whenever data is transmitted on a channel, the controller sends a current to an array of thermal elements embedded in the tapestry, converting data into heat and activating a thermochromic yarn woven into the tapestry. Like a Shroud of Turin, streams of data transmitted through a space appear as visual traces from an invisible dimension that gradually form and dissolve.
Exhibited at Centre Pompidou 2022
WifiTapestry is a dynamic wall hanging that visualises the wireless activity of a space. The tapestry visualises the ever changing "landscape" of radio frequencies around us. The invisible signals from Cellphones, printers and all kinds of smart devices leave an imprint as they try to negotiate available wireless channels. A controller listens to all traffic across 13 channels of the 2.4GHz WiFi Spectrum. $Whenever data is transmitted on a channel, the controller sends a current to an array of thermal elements embedded in the tapestry, converting data into heat and activating a thermochromic yarn woven into the tapestry. Like a Shroud of Turin, streams of data transmitted through a space appear as visual traces from an invisible dimension that gradually form and dissolve.
Exhibited at Centre Pompidou 2022
The Deleted City is a digital archaeology of the world wide web as it exploded into the 21st century. At that time the web was often described as an enormous digital library that you could visit or contribute to by building a homepage. The early citizens of the net (or netizens) took their netizenship serious, and built homepages about themselves and subjects they were experts in. These pioneers found their brave new world at Geocities, a free webhosting provider that was modelled after a city and where you could get a free "piece of land" to build your digital home in a certain neighbourhood based on the subject of your homepage. Heartland was – as a neigbourhood for all things rural – by far the largest, but there were neighbourhoods for fashion, arts and far east related topics to name just a few.$Around the turn of the century, Geocities had tens of millions of "homesteaders" as the digital tennants were called and was bought by Yahoo for three and a half billion dollars. Ten years later in 2009, as other metaphores of the internet (such as the social network) had taken over, and the homesteaders had left their properties vacant after migrating to Facebook, Geocities was shut down and deleted. In an heroic effort to preserve 10 years of collaborative work by 35 million people, the Archive Team made a backup of the site just before it shut down. The resulting 650 Gigabyte bittorrent file is the digital Pompeii that is the subject of an interactive excavation that allows you to wander through an episode of recent online history.
The Deleted City is a digital archaeology of the world wide web as it exploded into the 21st century. At that time the web was often described as an enormous digital library that you could visit or contribute to by building a homepage. The early citizens of the net (or netizens) took their netizenship serious, and built homepages about themselves and subjects they were experts in. These pioneers found their brave new world at Geocities, a free webhosting provider that was modelled after a city and where you could get a free "piece of land" to build your digital home in a certain neighbourhood based on the subject of your homepage. Heartland was – as a neigbourhood for all things rural – by far the largest, but there were neighbourhoods for fashion, arts and far east related topics to name just a few.$Around the turn of the century, Geocities had tens of millions of "homesteaders" as the digital tennants were called and was bought by Yahoo for three and a half billion dollars. Ten years later in 2009, as other metaphores of the internet (such as the social network) had taken over, and the homesteaders had left their properties vacant after migrating to Facebook, Geocities was shut down and deleted. In an heroic effort to preserve 10 years of collaborative work by 35 million people, the Archive Team made a backup of the site just before it shut down. The resulting 650 Gigabyte bittorrent file is the digital Pompeii that is the subject of an interactive excavation that allows you to wander through an episode of recent online history.
Interactive visualization for American artist James Turrell.
Deep inside Arizona's painted desert, lies Roden Crater, an extinct volcanic cinder cone. It is the site of a monumental artwork by James Turrell. Over the last 30 years, Turrell, famous for his installations concerning the perception of light, transformed the crater's eye into a naked-eye observatory that will eventually consist of 20 spaces each constructed to allow the observation of a specific portion of the sky or celestial event. The Interactive Celestial Map I made is part of the artists website and visualizes the relation between the sky above Roden Crater and the alignment of the skyspaces created by the artist. The interactive module shows the sky and the arrangement of the sun, the moon and the stars above Roden Crater at the current time displayed over a map of the crater.$Dragging a circular slider around the visualization allows you to go back and forth in time and observe how the celestial bodies align with the observation spaces in the crater at different points in time. The sun and moon rise and set, stars and planets move across the night sky and the summer an winter solstices mark the moments where the sun aligns with an opening in a space and projects it's image on a precisely positioned surface.
An auto-play button animates the sky in fast forward while a calendar allows you to see the sky at a specific date in the past or future.
The website was launched together with the opening of James Turrell: A Retrospective at the Los Angeles County Museum of Modern art.
Interactive visualization for American artist James Turrell.
Deep inside Arizona's painted desert, lies Roden Crater, an extinct volcanic cinder cone. It is the site of a monumental artwork by James Turrell. Over the last 30 years, Turrell, famous for his installations concerning the perception of light, transformed the crater's eye into a naked-eye observatory that will eventually consist of 20 spaces each constructed to allow the observation of a specific portion of the sky or celestial event. The Interactive Celestial Map I made is part of the artists website and visualizes the relation between the sky above Roden Crater and the alignment of the skyspaces created by the artist. The interactive module shows the sky and the arrangement of the sun, the moon and the stars above Roden Crater at the current time displayed over a map of the crater.$Dragging a circular slider around the visualization allows you to go back and forth in time and observe how the celestial bodies align with the observation spaces in the crater at different points in time. The sun and moon rise and set, stars and planets move across the night sky and the summer an winter solstices mark the moments where the sun aligns with an opening in a space and projects it's image on a precisely positioned surface.
An auto-play button animates the sky in fast forward while a calendar allows you to see the sky at a specific date in the past or future.
The website was launched together with the opening of James Turrell: A Retrospective at the Los Angeles County Museum of Modern art.
First Prize
An interactive datavisualization on 19.000 square feet of digital signboard on Times Square.
In March of 2002 NASA launched the GRACE mission. It consists of two satellites, designed to measure and map the Earth's gravity fields. Each month the two satellites complete a full scan of the earth, allowing scientists to study how variations in the earth's gravity fields — from which changes in groundwater levels can be derived — are developing over time. This 30 second data visualization uses the measurements collected by the GRACE satellites over a period of 10 years to show seasonal and longterm changes in groundwater levels. The Nasdaq screen shows a map of the world through the eyes of GRACE, a topography made of measurement data. It shows the yearly cycle of groundwater depletion and replenishment, the rainy seasons in the Amazon and parts of the world suffering from yearly droughts. These measurements allow us to see this natural spectacle on a global scale for the first time. However, they also reveal that some areas show a steady decline in groundwater levels. These longterm changes in groundwater levels are indicated$both on the map and on the narrow but very high screen of the Reuters building, where groundwater levels in several key areas are visualized as a virtual gauging rod. It shows that while some areas have been able to reverse the trend of declining groundwater levels, others show a sharp decline starting from the 1960's.
The aim of this visualization is to show on one hand the beauty and overwhelming complexity of the natural cycle of wet and dry seasons, and on the other hand highlight the challenge of carefully managing our use of groundwater.
An interactive feature allows the audience to engage with the visualization by adding their own city to a srcrolling ticker of historic groundwater levels across the world using a mobile application. The website headsup2012.com keeps an archive of all submitted cities.
The project was covered by The New York Times, Forbes, Fast Company, Infosthetics, NASA and National Geographic among others.
An interview about this project has been published by The Smithsonian
An interactive datavisualization on 19.000 square feet of digital signboard on Times Square.
In March of 2002 NASA launched the GRACE mission. It consists of two satellites, designed to measure and map the Earth's gravity fields. Each month the two satellites complete a full scan of the earth, allowing scientists to study how variations in the earth's gravity fields — from which changes in groundwater levels can be derived — are developing over time. This 30 second data visualization uses the measurements collected by the GRACE satellites over a period of 10 years to show seasonal and longterm changes in groundwater levels. The Nasdaq screen shows a map of the world through the eyes of GRACE, a topography made of measurement data. It shows the yearly cycle of groundwater depletion and replenishment, the rainy seasons in the Amazon and parts of the world suffering from yearly droughts. These measurements allow us to see this natural spectacle on a global scale for the first time. However, they also reveal that some areas show a steady decline in groundwater levels. These longterm changes in groundwater levels are indicated$both on the map and on the narrow but very high screen of the Reuters building, where groundwater levels in several key areas are visualized as a virtual gauging rod. It shows that while some areas have been able to reverse the trend of declining groundwater levels, others show a sharp decline starting from the 1960's.
The aim of this visualization is to show on one hand the beauty and overwhelming complexity of the natural cycle of wet and dry seasons, and on the other hand highlight the challenge of carefully managing our use of groundwater.
An interactive feature allows the audience to engage with the visualization by adding their own city to a srcrolling ticker of historic groundwater levels across the world using a mobile application. The website headsup2012.com keeps an archive of all submitted cities.
The project was covered by The New York Times, Forbes, Fast Company, Infosthetics, NASA and National Geographic among others.
An interview about this project has been published by The Smithsonian
2012 Finalist
The Deleted City is a digital archaeology of the world wide web as it exploded into the 21st century. At that time the web was often described as an enormous digital library that you could visit or contribute to by building a homepage. The early citizens of the net (or netizens) took their netizenship serious, and built homepages about themselves and subjects they were experts in.
These pioneers found their brave new world at Geocities, a free webhosting provider that was modelled after a city and where you could get a free "piece of land" to build your digital home in a certain neighbourhood based on the subject of your homepage. Heartland was – as a neigbourhood for all things rural – by far the largest, but there were neighbourhoods for fashion, arts and far east related topics to name just a few.$Around the turn of the century, Geocities had tens of millions of "homesteaders" as the digital tennants were called and was bought by Yahoo for three and a half billion dollars. Ten years later in 2009, as other metaphores of the internet (such as the social network) had taken over, and the homesteaders had left their properties vacant after migrating to Facebook, Geocities was shut down and deleted.
In an heroic effort to preserve 10 years of collaborative work by 35 million people, the Archive Team made a backup of the site just before it shut down. The resulting 650 Gigabyte bittorrent file is the digital Pompeii that is the subject of an interactive excavation that allows you to wander through an episode of recent online history.
The Deleted City is a digital archaeology of the world wide web as it exploded into the 21st century. At that time the web was often described as an enormous digital library that you could visit or contribute to by building a homepage. The early citizens of the net (or netizens) took their netizenship serious, and built homepages about themselves and subjects they were experts in.
These pioneers found their brave new world at Geocities, a free webhosting provider that was modelled after a city and where you could get a free "piece of land" to build your digital home in a certain neighbourhood based on the subject of your homepage. Heartland was – as a neigbourhood for all things rural – by far the largest, but there were neighbourhoods for fashion, arts and far east related topics to name just a few.$Around the turn of the century, Geocities had tens of millions of "homesteaders" as the digital tennants were called and was bought by Yahoo for three and a half billion dollars. Ten years later in 2009, as other metaphores of the internet (such as the social network) had taken over, and the homesteaders had left their properties vacant after migrating to Facebook, Geocities was shut down and deleted.
In an heroic effort to preserve 10 years of collaborative work by 35 million people, the Archive Team made a backup of the site just before it shut down. The resulting 650 Gigabyte bittorrent file is the digital Pompeii that is the subject of an interactive excavation that allows you to wander through an episode of recent online history.
Het Nieuwe Instituut’s archive is an extensive collection of objects, drawings and documentation related to the history of architecture in the Netherlands. Packed in boxes and sorted on shelves, the objects offer us a glimpse behind the scenes of architecture and the evolution of the Dutch urban landscape. Hidden in all these boxes lies a world of ideas: about forms, materials, people and the environment, a world of possibilities, successes and failures. A world that doesn’t reveal itself just like that. For anyone familiar with the structure of this archive it is a well-organised database of shelves and volumes. For the layperson it is an endless series of boxes.
The link between the boxes and the information they contain comes in the form of a digital file. You enter a search term in to the computer and the programme directs you to the right box. That’s assuming that you know what you’re looking for. For anyone with no knowledge whatsoever of architectural history there’s not much the computer can do to help you.
Data Volume Explorer is a proposal for a spatial, interactive search machine for anyone who doesn’t know what they’re looking for. The installation allows the user to play with the archive. Instead of boxes in endless rows of archive cupboards, the boxes can be arranged in endless configurations; chronologically, by architect, by format, or materials used.$A gigantic construction of boxes with drawings, or a seemingly never-ending landscape of boxes full of architectural models.
The space in which this experiment takes place is a virtual one. With Virtual Reality glasses the visitor can step out of the physical environment and enter the virtual archive. To make the transition clear the installation takes the form of a small space built from real archive boxes. As soon as you put the glasses on you see a virtual version of the same space.
Enter a search term and the space transforms in to a new environment based on a new configuration. You search using a (virtual) keyboard. As soon as you type a letter a list appears with possible search terms. By combining search terms you can filter the results, from the very broad to the very specific. The result could be a landscape of boxes with pencil drawings that reaches as far as the eye can see, or that one box full of sketches of Rotterdam’s Blaaktoren (The Pencil). If you linger at a certain box, it can be opened, and then the contents appear in the image. If you look away you return to the spatial environment. The starting point for the installation is the discovery of the archive. From very broad and almost random, to very specific. You can sort very specifically, with the box next to you full of drawings by the same architectural office, in the same city, or a model of the same material.
Het Nieuwe Instituut’s archive is an extensive collection of objects, drawings and documentation related to the history of architecture in the Netherlands. Packed in boxes and sorted on shelves, the objects offer us a glimpse behind the scenes of architecture and the evolution of the Dutch urban landscape. Hidden in all these boxes lies a world of ideas: about forms, materials, people and the environment, a world of possibilities, successes and failures. A world that doesn’t reveal itself just like that. For anyone familiar with the structure of this archive it is a well-organised database of shelves and volumes. For the layperson it is an endless series of boxes.
The link between the boxes and the information they contain comes in the form of a digital file. You enter a search term in to the computer and the programme directs you to the right box. That’s assuming that you know what you’re looking for. For anyone with no knowledge whatsoever of architectural history there’s not much the computer can do to help you.
Data Volume Explorer is a proposal for a spatial, interactive search machine for anyone who doesn’t know what they’re looking for. The installation allows the user to play with the archive. Instead of boxes in endless rows of archive cupboards, the boxes can be arranged in endless configurations; chronologically, by architect, by format, or materials used.$A gigantic construction of boxes with drawings, or a seemingly never-ending landscape of boxes full of architectural models.
The space in which this experiment takes place is a virtual one. With Virtual Reality glasses the visitor can step out of the physical environment and enter the virtual archive. To make the transition clear the installation takes the form of a small space built from real archive boxes. As soon as you put the glasses on you see a virtual version of the same space.
Enter a search term and the space transforms in to a new environment based on a new configuration. You search using a (virtual) keyboard. As soon as you type a letter a list appears with possible search terms. By combining search terms you can filter the results, from the very broad to the very specific. The result could be a landscape of boxes with pencil drawings that reaches as far as the eye can see, or that one box full of sketches of Rotterdam’s Blaaktoren (The Pencil). If you linger at a certain box, it can be opened, and then the contents appear in the image. If you look away you return to the spatial environment. The starting point for the installation is the discovery of the archive. From very broad and almost random, to very specific. You can sort very specifically, with the box next to you full of drawings by the same architectural office, in the same city, or a model of the same material.
Datavisualization for the Beijing Media Art Biennale 2016. The interactive visualization shows the curatorial framework and provides a theoretical and scientific context for the artworks on display.$Embedded in the program as both a web-app and an installation, the visualization functions as integral part of the exhibition.
Datavisualization for the Beijing Media Art Biennale 2016. The interactive visualization shows the curatorial framework and provides a theoretical and scientific context for the artworks on display.$Embedded in the program as both a web-app and an installation, the visualization functions as integral part of the exhibition.
Research for a data visualization. Taking six hours to complete, this four color fine liner plot$is the tactile outcome of a processing sketch that aims to visualize various international broadcasting networks.
Research for a data visualization. Taking six hours to complete, this four color fine liner plot$is the tactile outcome of a processing sketch that aims to visualize various international broadcasting networks.
