Editing article

Title

Summary

Content

<h3>Hey Gemini, explain me these pictures — in bash 🧑🏽‍💻</h3><a href="https://www.youtube.com/watch?v=jV1vkHv4zq8">Gemini is finally out</a>!And… it’s multimodal! Which means, you can use both images and text to ask for questions (the output is only text — for now 😉).Note: all the code snippets (and images) in here are available in my <a href="https://github.com/palladius/genai-googlecloud-scripts/tree/main/07-bash-gemini-multimodal-jam">GitHub repo</a>.Let’s get started!<h3>Hey Gemini, describe what you see..</h3>Let’s start by an album by my favourite band:<figure><img alt="" src="https://cdn-images-1.medium.com/max/536/0*gbubXqodziySrF_A" /><figcaption>One of my favourite albums of all time — will Gemini recognize it?</figcaption></figure><pre>$ ./gemini-generic.sh images/genesis-selling-england.jpg Describe what you see # 🤌 QUESTION: Describe what you see # 🌡️ TEMPERATURE: 0.2 # 👀 Examining image images/genesis-selling-england.jpg: JPEG image data, JFIF standard 1.01, resolution (DPI), density 96x96, segment length 16, baseline, precision 8, 536x528, components 3. # ♊ Gemini no Saga answer for you: The cover of Genesis album Selling England by the Pound features a painting by British artist Paul Whitehead. The painting depicts a group of people in a park, with a man sleeping on a bench in the foreground. The people are all wearing clothes from the 1920s or 1930s, and the painting has a nostalgic, almost surreal feel to it. The colors are muted and the figures are slightly blurred, which gives the painting a dreamlike quality. The painting is also full of symbolism, with the sleeping man representing England and the people around him representing the different aspects of English society. The painting has been interpreted in many different ways, but it is generally seen as a commentary on the state of England in the 1970s.</pre>♊ The cover of Genesis’ album Selling England by the Pound features a painting by British artist Paul Whitehead. The painting depicts a group of people in a park, with a man sleeping on a bench in the foreground. The people are all wearing clothes from the 1920s or 1930s, and the painting has a nostalgic, almost surreal feel to it. The colors are muted and the figures are slightly blurred, which gives the painting a dreamlike quality. The painting is also full of symbolism, with the sleeping man representing England and the people around him representing the different aspects of English society. The painting has been interpreted in many different ways, but it is generally seen as a commentary on the state of England in the 1970s.A quick googling confirms that <a href="https://en.wikipedia.org/wiki/Paul_Whitehead">Paul Whitehead</a> actually covered one of my favourite album of all times. If you love Genesis too and want to see me play Firth of Fifth, please check <a href="https://www.youtube.com/watch?v=4VBxd9n1dSU">my youtube video.</a>Let’s try this cat now:<figure><img alt="" src="https://cdn-images-1.medium.com/max/320/0*maiP4twRlm4le8Qx" /><figcaption>This is a cat from Google sample images</figcaption></figure>Gemini: A cat is walking in the snow. The cat is brown and white and has a long tail. The cat is looking at the camera. The snow is white and is covering the ground.<h4>How did I do it?</h4>If you check my bash script (<a href="https://github.com/palladius/genai-googlecloud-scripts/blob/main/07-bash-gemini-multimodal-jam/gemini-generic.sh">gemini-generic.sh</a>), you can see the request looks like this:<pre>data=$(_base64_encode_mac_or_linux &quot;$IMAGE&quot;) # Mac or Linux should both work! # ... cat &gt; &quot;$REQUEST_FILE&quot; &lt;&lt;EOF {&#39;contents&#39;: { &#39;role&#39;: &#39;USER&#39;, &#39;parts&#39;: [ {&#39;text&#39;: &#39;$QUESTION&#39;}, {&#39;inline_data&#39;: { &#39;data&#39;: &#39;$data&#39;, &#39;mime_type&#39;:&#39;image/jpeg&#39;}}] } } EOF</pre>The hardest part is actually the base64conversion, and making it work both on my Mac and on my Linux (as our grand-parents say in Italy “You wanted /bin/bash.? Now you pedal!”).<h4>An Italian image, explained in Italian</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/926/0*KyG34HB0fyBRdHH8" /><figcaption>The city of XXX, in Italy. Seen from the YYY palace (surprise!). Picture from my colleague Daria.</figcaption></figure><pre>./gemini-explain-image.sh images/italian-town.jpg [..] This is a view of the city of Trento, Italy from the Buonconsiglio Castle.</pre>Gemini (🇬🇧): This is a view of the city of Trento, Italy from the Buonconsiglio Castle.Let’s ask the same in Italian:<pre>$ GENERATE_MP3=true ./gemini-explain-image-italian.sh images/italian-town.jpg # 🤌 QUESTION: Descrivimi cosa vedi in questa immagine # 🌡️ TEMPERATURE: 0.2 # 👀 Examining image images/italian-town.jpg: JPEG image data, JFIF standard 1.01, aspect ratio, density 1x1, segment length 16, Exif Standard: [TIFF image data, little-endian, direntries=1, software=Google], baseline, precision 8, 926x1230, components 3. # ♊ Gemini no Saga answer for you: La foto mostra una loggia con delle colonne in pietra che incorniciano la vista di una città. La città è circondata da montagne e si possono vedere i tetti delle case e le torri delle chiese. Il cielo è azzurro e ci sono delle nuvole bianche. # TTS_LANG: it-IT Written .tmp.tts-output.json. curl_ret=0 t.audio.encoded: ASCII text, with very long lines (65536), with no line terminators t.mp3: MPEG ADTS, layer III, v2, 32 kbps, 24 kHz, Monaural t.mp3: MPEG ADTS, layer III, v2, 32 kbps, 24 kHz, Monaural All good. MP3 created: &#39;t.La foto mostra una loggia con delle colonne in pie.mp3&#39;</pre>Gemini (🇮🇹): La foto mostra una loggia con delle colonne in pietra che incorniciano la vista di una città. La città è circondata da montagne e si possono vedere i tetti delle case e le torri delle chiese. Il cielo è azzurro e ci sono delle nuvole bianche.As you see, Italian is more verbose (my wife says the same about me!) but it’s not aware of the <a href="https://www.buonconsiglio.it/">Buonconsiglio Palace</a>. Interesting! I presume the Italian model has less training material to learn from than the English one.Makes sense.<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*kM6xWj1LtmR46I0q" /><figcaption>Btw, I highly recommend Trento, I was cycling around there: great views and great wines!</figcaption></figure><h4>Comparing two images</h4>How about comparing two images?<figure><img alt="" src="https://cdn-images-1.medium.com/max/786/1*yiGN2IyD-eaLglpR63_Krw.png" /><figcaption>Gemini constellation vs Saint Seiya’s Gemini saint.</figcaption></figure>Some sample code is in my Makefile for convenience. try this:<pre>ricc@derek:~/git/genai-googlecloud-scripts/07-bash-gemini-multimodal-jam$ 🐼 !mak make compare-two-geminis ./gemini-generic-two-pics.sh images/gemini-constellation.png images/saga-blue-hair.jpg ♊️ Question: Can you highlight similarity and differences between the two? Also, do you recognize the same person in both of them? 👀 Examining image1 images/gemini-constellation.png: images/gemini-constellation.png: PNG image data, 1675 x 1302, 8-bit/color RGBA, non-interlaced. 👀 Examining image2 images/saga-blue-hair.jpg: images/saga-blue-hair.jpg: JPEG image data, JFIF standard 1.01, aspect ratio, density 1x1, segment length 16, baseline, precision 8, 193x261, components 3. ♊️ Describing attached image: The two images are of the constellation Gemini and the anime character Gemini Saga. The constellation is said to represent the twins Castor and Pollux, while the character Gemini Saga is a Gemini Saint in the anime series Saint Seiya. Both images depict two figures that are connected to each other. The constellation is made up of stars, while the character is a human.</pre>♊ Response: ️ The two images are of the constellation Gemini and the anime character Gemini Saga. The constellation is said to represent the twins Castor and Pollux, while the character Gemini Saga is a Gemini Saint in the anime series Saint Seiya. Both images depict two figures that are connected to each other. The constellation is made up of stars, while the character is a human.Today we’ve learnt something we certainly didn’t know: a constellation is made of stars, while the guy on the right is human 😃<h4>Introducing Audio!</h4>Why don’t we throw some audio in the mix?I’ve added a tts.sh , which creates an MP3 out of an english (or Italian!) text given in ARGV. Convenient uh?<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*-QOBFPt1GsVq2LVK" /><figcaption>My kids beside American Santa Klaus (we live in Zurich, there’s a Santa for every language and ethnicity!)</figcaption></figure>A few weeks ago my kids went to visit Santa Klaus.. the American Santa — they’ll tell you there’s another who speaks German instead, he’s called Samiklaus, they still can’t decide if its the same guy — my kids are X and Y year old; how old? Let’s ask Gemini!<pre>$ make age-test # equivalent to: $ GENERATE_MP3=true ./gemini-generic.sh images/ricc-family-with-santa.jpg Tell me the age of the people you see, from left to right. # 🤌 QUESTION: Tell me the age of the people you see, from left to right. # 🌡️ TEMPERATURE: 0.2 # 👀 Examining image images/ricc-family-with-santa.jpg: JPEG image data, JFIF standard 1.01, aspect ratio, density 1x1, segment length 16, Exif Standard: [TIFF image data, little-endian, direntries=3, software=Google], baseline, precision 8, 1164x826, components 3. # ♊ Gemini no Saga answer for you: 1. 30-35 2. 2-3 3. 40-45 4. 2-3 5. 60-65 [..] All good. MP3 created [..]</pre>Now I’m older than 40–45 (46) so technically speaking this is a mistake, but as a human interaction I will consider it as a success 😉Now this is not the best of MP3s, but it works! Check out <a href="https://github.com/palladius/genai-googlecloud-scripts/blob/main/07-bash-gemini-multimodal-jam/images/ricc-family-with-santa.jpg.mp3">ricc-family-with-santa.jpg.mp3</a>.You can also find Trento in Italian here: <a href="https://github.com/palladius/genai-googlecloud-scripts/blob/main/07-bash-gemini-multimodal-jam/images/italian-town.jpg.mp3">italian-town.jpg.mp3</a>.<h4>Something useful now: understand a diagram!</h4>A colleague told me yesterday: the power of Gemini is that not only “he” understands text and images, also the very complex relationship between items in a diagram. This morning my Accrux earphones wouldn’t work, so I decided to understand how they work. I downloaded the <a href="https://raw.githubusercontent.com/palladius/genai-googlecloud-scripts/main/07-bash-gemini-multimodal-jam/images/instruction-manuals/Acrux-User-Manual-4700503.pdf">User Manual</a>, uploaded to Gemini <a href="https://raw.githubusercontent.com/palladius/genai-googlecloud-scripts/main/07-bash-gemini-multimodal-jam/images/instruction-manuals/Acrux-User-Manual-4700503.png">as image</a>, and asked Gemini a few questions.<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*hfF-icWAS9U4T30e.png" /><figcaption>I’m not so young anymore so I can’t read this unless it’s printed on A3. And I respect Amazon too much!</figcaption></figure>From UI, this is easy peasy. Upload image and ask question:<figure><img alt="" src="https://cdn-images-1.medium.com/max/612/0*TXJPDl-xEJFt6FEL" /></figure>And what about bash? Let’s do the same from CLI:<pre>$ make read-instruction-manual-for-me [..] ./gemini-generic.sh images/instruction-manuals/Acrux-User-Manual-4700503.png &#39;1. How do i TURN it on? 2. Where is the power button located? 3. Is this the one called ANC?&#39; # 🤌 QUESTION: 1. How do i TURN it on? 2. Where is the power button located? 3. Is this the one called ANC? # 🌡️ TEMPERATURE: 0.2 # 👀 Examining image images/instruction-manuals/Acrux-User-Manual-4700503.png: PNG image data, 1664 x 929, 8-bit/color RGBA, non-interlaced. # ♊ Gemini no Saga answer for you: 1. Long press the power button for 2 seconds. 2. The power button is located on the right earcup. 3. Yes, this is the one called ANC. # Note: No mp3 file generated (use GENERATE_MP3=true to generate one)</pre>My question (I actually asked the questions one by one, so think of it as an avalanche-chat):<ol><li>How do i TURN it on?</li><li>Where is the power button located?</li><li>Is this the one called ANC?</li></ol># ♊ Gemini answer: 1. Long press the power button for 2 seconds. 2. The power button is located on the right earcup. 3. Yes, this is the one called ANC.Bingo!<h4>An unexpected Games of Thrones plot twist</h4>This is the avatar I use in Google. I’m really curious to see what gemini can infer from here, also because I used some filters.<figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*w_qhJiiXjD2ZudEZ" /><figcaption>Riccardo in the Amsterdam office, a few years/kilos ago.</figcaption></figure><pre>$ ./gemini-explain-image.sh images/ricc-logo.png [..] This is a photo of a man standing behind a Google Cloud Platform cutout. The man is smiling and wearing a shirt that says, “That’s what I do, I drink and I know things.” The background is a brick wall with blue and white accents.</pre>♊ Gemini: This is a photo of a man standing behind a Google Cloud Platform cutout. The man is smiling and wearing a shirt that says, “That’s what I do, I drink and I know things.” The background is a brick wall with blue and white accents.And I thought! Of course, this is my favourite Games of Thrones t-shirt. Let’s see if he knows:Q: Do you recognize the quote in this person tshirt?<pre>$ GENERATE_MP3=true ./gemini-generic.sh images/ricc-logo.png Do you recognize the quote in this person tshirt [..] &quot;That&#39;s what I do, I drink and I know things&quot; is a quote from the TV show Game of Thrones, said by the character Tyrion Lannister.</pre>♊ Response: “That’s what I do, I drink and I know things” is a quote from the TV show Game of Thrones, said by the character Tyrion Lannister.And you also have the MP3 if you wish: <a href="https://github.com/palladius/genai-googlecloud-scripts/blob/main/07-bash-gemini-multimodal-jam/images/ricc-logo.png.mp3">images/ricc-logo.png.mp3</a><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*gz3JRSvt3G_-iDcm" /></figure>Chapeau, Gemini!<h4>Conclusions</h4>Calling Gemini is easy, fun and powerful!<ul><li>Easy. If you can invoke Gemini via curl, imagine how easy it is to invoke it with your favourite language! <a href="https://cloud.google.com/vertex-ai/docs/generative-ai/start/quickstarts/quickstart-multimodal">Try out our SDK</a> in Python, Node, Java, Go. For everything else there’s REST/cURL!</li><li>Fun. look at how many use cases I’ve shown in this page: comparing two images, asking questions abuot an image, a writing, age of people, .. possibilities are endless.</li><li>Powerful. Multi-modal is a game changer. And for what I hear, more modes are coming. 😉</li></ul>Did you like this? Want more of this? Write it in the comments!Riccardo<img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=06c03d0d0512" width="1" height="1" alt=""><hr><a href="https://blog.devops.dev/hey-gemini-explain-me-these-pictures-in-bash-06c03d0d0512">Hey Gemini, explain me these pictures — in bash 🧑🏽‍💻</a> was originally published in <a href="https://blog.devops.dev">DevOps.dev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.

Author

Link

Published date

Image url

Feed url

Guid

Hidden blurb

--- !ruby/object:Feedjira::Parser::RSSEntry
title: Hey Gemini, explain me these pictures — in bash ‍
url: https://blog.devops.dev/hey-gemini-explain-me-these-pictures-in-bash-06c03d0d0512?source=rss-b5293b96912f------2
author: Riccardo Carlesso
categories:
- bash
- gemini
- curl
- genai
- google
published: 2023-12-14 15:16:16.000000000 Z
entry_id: !ruby/object:Feedjira::Parser::GloballyUniqueIdentifier
 is_perma_link: 'false'
 guid: https://medium.com/p/06c03d0d0512
carlessian_info:
 news_filer_version: 2
 newspaper: Riccardo Carlesso - Medium
 macro_region: Blogs
rss_fields:
- title
- url
- author
- categories
- published
- entry_id
- content
content: "<h3>Hey Gemini, explain me these pictures — in bash \U0001F9D1\U0001F3FD‍\U0001F4BB</h3><a
 href=\"https://www.youtube.com/watch?v=jV1vkHv4zq8\">Gemini is
 finally out</a>!And… it’s multimodal! Which means, you can
 use both images and text to ask for questions (the output is only text — for now \U0001F609).Note:
 all the code snippets (and images) in here are available in my <a href=\"https://github.com/palladius/genai-googlecloud-scripts/tree/main/07-bash-gemini-multimodal-jam\">GitHub repo</a>.Let’s
 get started!<h3>Hey Gemini, describe what you see..</h3>Let’s start by an
 album by my favourite band:<figure><img alt=\"\" src=\"https://cdn-images-1.medium.com/max/536/0*gbubXqodziySrF_A\"
 /><figcaption>One of my favourite albums of all time — will Gemini recognize it?</figcaption></figure><pre>$
 ./gemini-generic.sh images/genesis-selling-england.jpg Describe what you see #
 \U0001F90C QUESTION: Describe what you see # \U0001F321️ TEMPERATURE: 0.2 #
 \U0001F440 Examining image images/genesis-selling-england.jpg: JPEG image data,
 JFIF standard 1.01, resolution (DPI), density 96x96, segment length 16, baseline,
 precision 8, 536x528, components 3. # ♊ Gemini no Saga answer for you: The
 cover of Genesis album Selling England by the Pound features a painting by British
 artist Paul Whitehead. The painting depicts a group of people in a park, with
 a man sleeping on a bench in the foreground. The people are all wearing clothes
 from the 1920s or 1930s, and the painting has a nostalgic, almost surreal feel
 to it. The colors are muted and the figures are slightly blurred, which gives
 the painting a dreamlike quality. The painting is also full of symbolism, with
 the sleeping man representing England and the people around him representing
 the different aspects of English society. The painting has been interpreted in
 many different ways, but it is generally seen as a commentary on the state of
 England in the 1970s.</pre>♊ The cover of Genesis’ album Selling England
 by the Pound features a painting by British artist Paul Whitehead. The painting
 depicts a group of people in a park, with a man sleeping on a bench in the foreground.
 The people are all wearing clothes from the 1920s or 1930s, and the painting has
 a nostalgic, almost surreal feel to it. The colors are muted and the figures are
 slightly blurred, which gives the painting a dreamlike quality. The painting is
 also full of symbolism, with the sleeping man representing England and the people
 around him representing the different aspects of English society. The painting has
 been interpreted in many different ways, but it is generally seen as a commentary
 on the state of England in the 1970s.A quick googling confirms that
 <a href=\"https://en.wikipedia.org/wiki/Paul_Whitehead\">Paul Whitehead</a> actually
 covered one of my favourite album of all times. If you love Genesis too and want
 to see me play Firth of Fifth, please check <a href=\"https://www.youtube.com/watch?v=4VBxd9n1dSU\">my
 youtube video.</a>Let’s try this cat now:<figure><img alt=\"\" src=\"https://cdn-images-1.medium.com/max/320/0*maiP4twRlm4le8Qx\"
 /><figcaption>This is a cat from Google sample images</figcaption></figure>Gemini:
 A cat is walking in the snow. The cat is brown and white and has a long tail. The
 cat is looking at the camera. The snow is white and is covering the ground.<h4>How
 did I do it?</h4>If you check my bash script (<a href=\"https://github.com/palladius/genai-googlecloud-scripts/blob/main/07-bash-gemini-multimodal-jam/gemini-generic.sh\">gemini-generic.sh</a>),
 you can see the request looks like this:<pre>data=$(_base64_encode_mac_or_linux
 &quot;$IMAGE&quot;) # Mac or Linux should both work! # ... cat &gt; &quot;$REQUEST_FILE&quot;
 &lt;&lt;EOF {&#39;contents&#39;: { &#39;role&#39;: &#39;USER&#39;, 
 \ &#39;parts&#39;: [ {&#39;text&#39;: &#39;$QUESTION&#39;}, {&#39;inline_data&#39;:
 { &#39;data&#39;: &#39;$data&#39;, &#39;mime_type&#39;:&#39;image/jpeg&#39;}}] 
 \ } } EOF</pre>The hardest part is actually the base64conversion,
 and making it work both on my Mac and on my Linux (as our grand-parents say in Italy
 “You wanted /bin/bash.? Now you pedal!”).<h4>An Italian image,
 explained in Italian</h4><figure><img alt=\"\" src=\"https://cdn-images-1.medium.com/max/926/0*KyG34HB0fyBRdHH8\"
 /><figcaption>The city of XXX, in Italy. Seen from the YYY palace (surprise!). Picture
 from my colleague Daria.</figcaption></figure><pre>./gemini-explain-image.sh images/italian-town.jpg [..] 
 This is a view of the city of Trento, Italy from the Buonconsiglio Castle.</pre>Gemini
 (\U0001F1EC\U0001F1E7): This is a view of
 the city of Trento, Italy from the Buonconsiglio Castle.Let’s ask the
 same in Italian:<pre>$ GENERATE_MP3=true ./gemini-explain-image-italian.sh images/italian-town.jpg #
 \U0001F90C QUESTION: Descrivimi cosa vedi in questa immagine # \U0001F321️ TEMPERATURE:
 0.2 # \U0001F440 Examining image images/italian-town.jpg: JPEG image data, JFIF
 standard 1.01, aspect ratio, density 1x1, segment length 16, Exif Standard: [TIFF
 image data, little-endian, direntries=1, software=Google], baseline, precision 8,
 926x1230, components 3. # ♊ Gemini no Saga answer for you: La foto mostra
 una loggia con delle colonne in pietra che incorniciano la vista di una città. 
 La città è circondata da montagne e si possono vedere i tetti delle case e le torri
 delle chiese. Il cielo è azzurro e ci sono delle nuvole bianche. # TTS_LANG:
 it-IT Written .tmp.tts-output.json. curl_ret=0 t.audio.encoded: ASCII text,
 with very long lines (65536), with no line terminators t.mp3: MPEG
 ADTS, layer III, v2, 32 kbps, 24 kHz, Monaural t.mp3: MPEG ADTS, layer III,
 v2, 32 kbps, 24 kHz, Monaural All good. MP3 created: &#39;t.La foto mostra una
 loggia con delle colonne in pie.mp3&#39;</pre>Gemini (\U0001F1EE\U0001F1F9):
 La foto mostra una loggia con delle colonne in pietra che incorniciano la vista
 di una città. La città è circondata da montagne e si possono vedere i tetti delle
 case e le torri delle chiese. Il cielo è azzurro e ci sono delle nuvole bianche.As
 you see, Italian is more verbose (my wife says the same about me!) but it’s not
 aware of the <a href=\"https://www.buonconsiglio.it/\">Buonconsiglio Palace</a>.
 Interesting! I presume the Italian model has less training material to learn from
 than the English one.Makes sense.<figure><img alt=\"\" src=\"https://cdn-images-1.medium.com/max/1024/0*kM6xWj1LtmR46I0q\"
 /><figcaption>Btw, I highly recommend Trento, I was cycling around there: great
 views and great wines!</figcaption></figure><h4>Comparing two images</h4>How
 about comparing two images?<figure><img alt=\"\" src=\"https://cdn-images-1.medium.com/max/786/1*yiGN2IyD-eaLglpR63_Krw.png\"
 /><figcaption>Gemini constellation vs Saint Seiya’s Gemini saint.</figcaption></figure>Some
 sample code is in my Makefile for convenience. try this:<pre>ricc@derek:~/git/genai-googlecloud-scripts/07-bash-gemini-multimodal-jam$
 \U0001F43C !mak make compare-two-geminis ./gemini-generic-two-pics.sh images/gemini-constellation.png
 \ images/saga-blue-hair.jpg ♊️ Question: Can you highlight similarity and differences
 between the two? Also, do you recognize the same person in both of them? \U0001F440
 Examining image1 images/gemini-constellation.png: images/gemini-constellation.png:
 PNG image data, 1675 x 1302, 8-bit/color RGBA, non-interlaced. \U0001F440 Examining
 image2 images/saga-blue-hair.jpg: images/saga-blue-hair.jpg: JPEG image data, JFIF
 standard 1.01, aspect ratio, density 1x1, segment length 16, baseline, precision
 8, 193x261, components 3. ♊️ Describing attached image: The two images are
 of the constellation Gemini and the anime character Gemini Saga. The constellation
 is said to represent the twins Castor and Pollux, while the character Gemini Saga
 is a Gemini Saint in the anime series Saint Seiya. Both images depict two figures
 that are connected to each other. The constellation is made up of stars, while the
 character is a human.</pre>♊ Response: ️ The two images
 are of the constellation Gemini and the anime character Gemini Saga. The constellation
 is said to represent the twins Castor and Pollux, while the character Gemini Saga
 is a Gemini Saint in the anime series Saint Seiya. Both images depict two figures
 that are connected to each other. The constellation is made up of stars, while the
 character is a human.Today we’ve learnt something we certainly didn’t
 know: a constellation is made of stars, while the guy on the right is human \U0001F603<h4>Introducing
 Audio!</h4>Why don’t we throw some audio in the mix?I’ve added a tts.sh ,
 which creates an MP3 out of an english (or Italian!) text given in ARGV. Convenient
 uh?<figure><img alt=\"\" src=\"https://cdn-images-1.medium.com/max/1024/0*-QOBFPt1GsVq2LVK\"
 /><figcaption>My kids beside American Santa Klaus (we live in Zurich, there’s a
 Santa for every language and ethnicity!)</figcaption></figure>A few weeks ago
 my kids went to visit Santa Klaus.. the American Santa — they’ll tell you there’s
 another who speaks German instead, he’s called Samiklaus, they still can’t
 decide if its the same guy — my kids are X and Y year old; how old? Let’s ask Gemini!<pre>$
 make age-test # equivalent to: $ GENERATE_MP3=true ./gemini-generic.sh images/ricc-family-with-santa.jpg
 Tell me the age of the people you see, from left to right. # \U0001F90C QUESTION:
 Tell me the age of the people you see, from left to right. # \U0001F321️ TEMPERATURE:
 0.2 # \U0001F440 Examining image images/ricc-family-with-santa.jpg: JPEG image
 data, JFIF standard 1.01, aspect ratio, density 1x1, segment length 16, Exif Standard:
 [TIFF image data, little-endian, direntries=3, software=Google], baseline, precision
 8, 1164x826, components 3. # ♊ Gemini no Saga answer for you: 1. 30-35 2.
 2-3 3. 40-45 4. 2-3 5. 60-65 [..] All good. MP3 created [..]</pre>Now
 I’m older than 40–45 (46) so technically speaking this is a mistake, but as a human
 interaction I will consider it as a success \U0001F609Now this is not the
 best of MP3s, but it works! Check out <a href=\"https://github.com/palladius/genai-googlecloud-scripts/blob/main/07-bash-gemini-multimodal-jam/images/ricc-family-with-santa.jpg.mp3\">ricc-family-with-santa.jpg.mp3</a>.You
 can also find Trento in Italian here: <a href=\"https://github.com/palladius/genai-googlecloud-scripts/blob/main/07-bash-gemini-multimodal-jam/images/italian-town.jpg.mp3\">italian-town.jpg.mp3</a>.<h4>Something
 useful now: understand a diagram!</h4>A colleague told me yesterday: the power
 of Gemini is that not only “he” understands text and images, also the very complex
 relationship between items in a diagram. This morning my Accrux earphones wouldn’t
 work, so I decided to understand how they work. I downloaded the <a href=\"https://raw.githubusercontent.com/palladius/genai-googlecloud-scripts/main/07-bash-gemini-multimodal-jam/images/instruction-manuals/Acrux-User-Manual-4700503.pdf\">User
 Manual</a>, uploaded to Gemini <a href=\"https://raw.githubusercontent.com/palladius/genai-googlecloud-scripts/main/07-bash-gemini-multimodal-jam/images/instruction-manuals/Acrux-User-Manual-4700503.png\">as
 image</a>, and asked Gemini a few questions.<figure><img alt=\"\" src=\"https://cdn-images-1.medium.com/max/1024/0*hfF-icWAS9U4T30e.png\"
 /><figcaption>I’m not so young anymore so I can’t read this unless it’s printed
 on A3. And I respect Amazon too much!</figcaption></figure>From UI, this is easy
 peasy. Upload image and ask question:<figure><img alt=\"\" src=\"https://cdn-images-1.medium.com/max/612/0*TXJPDl-xEJFt6FEL\"
 /></figure>And what about bash? Let’s do the same from CLI:<pre>$ make read-instruction-manual-for-me
 [..] ./gemini-generic.sh images/instruction-manuals/Acrux-User-Manual-4700503.png
 &#39;1. How do i TURN it on? 2. Where is the power button located? 3. Is this the
 one called ANC?&#39; # \U0001F90C QUESTION: 1. How do i TURN it on? 2. Where
 is the power button located? 3. Is this the one called ANC? # \U0001F321️ TEMPERATURE:
 0.2 # \U0001F440 Examining image images/instruction-manuals/Acrux-User-Manual-4700503.png:
 PNG image data, 1664 x 929, 8-bit/color RGBA, non-interlaced. # ♊ Gemini no
 Saga answer for you: 1. Long press the power button for 2 seconds. 2. The
 power button is located on the right earcup. 3. Yes, this is the one called ANC. #
 Note: No mp3 file generated (use GENERATE_MP3=true to generate one)</pre>My question
 (I actually asked the questions one by one, so think of it as an avalanche-chat):<ol><li>How
 do i TURN it on?</li><li>Where is the power button located?</li><li>Is this the
 one called ANC?</li></ol># ♊ Gemini answer: 1. Long press the power button
 for 2 seconds. 2. The power button is located on the right earcup. 3. Yes,
 this is the one called ANC.Bingo!<h4>An unexpected Games of Thrones
 plot twist</h4>This is the avatar I use in Google. I’m really curious to see
 what gemini can infer from here, also because I used some filters.<figure><img
 alt=\"\" src=\"https://cdn-images-1.medium.com/max/600/0*w_qhJiiXjD2ZudEZ\" /><figcaption>Riccardo
 in the Amsterdam office, a few years/kilos ago.</figcaption></figure><pre>$ ./gemini-explain-image.sh
 images/ricc-logo.png [..] This is a photo of a man standing behind a Google
 Cloud Platform cutout. The man is smiling and wearing a shirt that says, “That’s
 what I do, I drink and I know things.” The background is a brick wall with blue and
 white accents.</pre>♊ Gemini: This is a photo of a man standing
 behind a Google Cloud Platform cutout. The man is smiling and wearing a shirt that
 says, “That’s what I do, I drink and I know things.” The background is a brick
 wall with blue and white accents.And I thought! Of course, this is
 my favourite Games of Thrones t-shirt. Let’s see if he knows:Q:
 Do you recognize the quote in this person tshirt?<pre>$ GENERATE_MP3=true
 ./gemini-generic.sh images/ricc-logo.png Do you recognize the quote in this person
 tshirt [..] &quot;That&#39;s what I do, I drink and I know things&quot; is
 a quote from the TV show Game of Thrones, said by the character Tyrion Lannister.</pre>♊
 Response: “That’s what I do, I drink and I know things” is a quote
 from the TV show Game of Thrones, said by the character Tyrion Lannister.And
 you also have the MP3 if you wish: <a href=\"https://github.com/palladius/genai-googlecloud-scripts/blob/main/07-bash-gemini-multimodal-jam/images/ricc-logo.png.mp3\">images/ricc-logo.png.mp3</a><figure><img
 alt=\"\" src=\"https://cdn-images-1.medium.com/max/1000/0*gz3JRSvt3G_-iDcm\" /></figure>Chapeau,
 Gemini!<h4>Conclusions</h4>Calling Gemini is easy, fun and powerful!<ul><li>Easy.
 If you can invoke Gemini via curl, imagine how easy it is to invoke it with your
 favourite language! <a href=\"https://cloud.google.com/vertex-ai/docs/generative-ai/start/quickstarts/quickstart-multimodal\">Try
 out our SDK</a> in Python, Node, Java, Go. For everything
 else there’s REST/cURL!</li><li>Fun. look at how many use cases
 I’ve shown in this page: comparing two images, asking questions abuot an image,
 a writing, age of people, .. possibilities are endless.</li><li>Powerful.
 Multi-modal is a game changer. And for what I hear, more modes are coming. \U0001F609</li></ul>Did
 you like this? Want more of this? Write it in the comments!Riccardo<img
 src=\"https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=06c03d0d0512\"
 width=\"1\" height=\"1\" alt=\"\"><hr><a href=\"https://blog.devops.dev/hey-gemini-explain-me-these-pictures-in-bash-06c03d0d0512\">Hey
 Gemini, explain me these pictures — in bash \U0001F9D1\U0001F3FD‍\U0001F4BB</a>
 was originally published in <a href=\"https://blog.devops.dev\">DevOps.dev</a> on
 Medium, where people are continuing the conversation by highlighting and responding
 to this story."

Language

Active

Ricc internal notes

Ricc source

Show this article Back to articles