THE TRANSMISSION CHAIN EXPERIMENT

Chinese whisper is a classic popular game of message relay and the entertainment comes from the comparison of the initial message and the final message and the goal is to have the closest deviation as possible. We would like to test out this experiment on machines and how communication happens between two “entities”. We would like to experiment on the communication between a text to image AI model like DALL-E 2 and an image to text AI model (CLIP to Image) and see how they communication happens through the latent space understanding of both entities together with human intervention

DALL•E AI link: 

For our ‘TEXT-TO-IMAGE’ AI were using DALL-E 2. DALL·E 2 is an AI that can create original, realistic images and art from a
text description. It can combine concepts, attributes, and styles.

And for our ‘IMAGE-TO-TEXT’ we are using model named CLIP that use the coco AI. An AI used for object detection and with GPT-2 AI and converts those images to a text description.

For our Methodology, We would test how we as humans communicate with MACHINES and how do machines communicate with each other. It is also an exercise of how we explore the latent space. To test how a MACHINE is speaking to another MACHINE, we’ve set up an initial prompt that would be fed into the DALL-E AI, then after generating an image it would be interpreted by the CLIP AI that translates an image to text description and loops onwards and we would see after a few iterations what comes out and how the machines interprets the initial message.

HUMAN and AI

Case Study 1: Artistic Directors

Case Study 2: Architects

To dive in the usages of ai, we took a case study of artistic directors and architects both communicating using prompts to a machine

The artist duo created an instagram page on which they showcase the prompts made by dalle

This prompt is of hamster showing us its sophisticated green schoolbag, catalog photo.

And this one is still of a pink panther thats super realistic in Twin Peaks Movie

Oh a happy clown puppet swimming in a lake, although we taught that these pictures are a bit disturbing

For architect, the usage of words such as octane render seem to give some resolution to the image

AI and AI

EXPERIMENT 1_CAT EATING LUNCH WITH A FISH

in our first experiment we gave dall-e a promt about cat eating with fish which it generated an image that was somewhat accurate but prompts that followed continued to transform and change persevering the initial message of cat eating while adding elements like table

The interpretations are more or less the same between the two ai models, but they both misunderstood the human input

EXPERIMENT 2_CROWD RUNNING THROUGH A COURTYARD

in our second experiment we gave Dall-e a prompt about  a crowd  running through a courtyard  which it generated an image that was somewhat accurate but prompts that followed  continued to transform and change diverging and transforming completely  as we can see the three iteration  and final output

The Information transforms completely in the conversation between AI and AI. the only features preserved were outdoors and crowd

CONCLUSIONS

• A new linguistic bias happening between Humans and AI.
• The latent space explorations give different explanations and meaning to the new images.
• From the experiments;
  • Information is lost in the sequence of communication from human ai and i to ai.
  • The longer the communication chain is, the greater is the deviation.
  • The Latent space visuals by the AI model is similar to the human imagination and how accurate it the interpretation might be.
  • In our experiments we discovered that meaning information is lost and transformed as the communication chain extends.

The Transmission Chain Experiment is a project of IAAC, Institute for Advanced Architecture of Catalonia developed in the Master of Advanced Computation in Architecture and Design 2021/22 by Students: Neil John Bersabe, Daniyal Tariq, Mahmoud Ramdane Faculty: Zeynep Aksöz, Mark Balzar