Editing article

Title

Summary

Content

<figure><img alt="Logo of the Gemini large language model launched by Google" src="https://cdn-images-1.medium.com/max/1024/0*4ohfuLfaP-ZAbo5_" /></figure>Google announced today the availability of <a href="https://cloud.google.com/blog/products/ai-machine-learning/gemini-support-on-vertex-ai">Gemini</a>, its latest and more powerful Large Language Model. Gemini is multimodal, which means it’s able to consume not only text, but also images or videos.I had the pleasure of working on the Java samples and help with the Java SDK, with wonderful engineer colleagues, and I’d like to share some examples of what you can do with Gemini, using Java!First of all, you’ll need to have an account on Google Cloud and created a project. The Vertex AI API should be enabled, to be able to access the Generative AI services, and in particular the Gemini large language model. Be sure to check out the <a href="https://cloud.google.com/vertex-ai/docs/generative-ai/start/quickstarts/quickstart-multimodal?hl=en">instructions</a>.<h3>Preparing your project build</h3>To get started with some coding, you’ll need to create a Gradle or a Maven build file that requires the Google Cloud libraries BOM, and the google-cloud-vertexai library. Here&#39;s an example with Maven:<pre>... &lt;dependencyManagement&gt; &lt;dependencies&gt; &lt;dependency&gt; &lt;artifactId&gt;libraries-bom&lt;/artifactId&gt; &lt;groupId&gt;com.google.cloud&lt;/groupId&gt; &lt;scope&gt;import&lt;/scope&gt; &lt;type&gt;pom&lt;/type&gt; &lt;version&gt;26.29.0&lt;/version&gt; &lt;/dependency&gt; &lt;/dependencies&gt; &lt;/dependencyManagement&gt; &lt;dependencies&gt; &lt;dependency&gt; &lt;groupId&gt;com.google.cloud&lt;/groupId&gt; &lt;artifactId&gt;google-cloud-vertexai&lt;/artifactId&gt; &lt;/dependency&gt; ... &lt;/dependencies&gt; ...</pre><h3>Your first queries</h3>Now let’s have a look at our first multimodal example, mixing text prompts and images:<pre>try (VertexAI vertexAI = new VertexAI(projectId, location)) { byte[] imageBytes = Base64.getDecoder().decode(dataImageBase64); GenerativeModel model = new GenerativeModel(&quot;gemini-pro-vision&quot;, vertexAI); GenerateContentResponse response = model.generateContent( ContentMaker.fromMultiModalData( &quot;What is this image about?&quot;, PartMaker.fromMimeTypeAndData(&quot;image/jpg&quot;, imageBytes) )); System.out.println(ResponseHandler.getText(response)); }</pre>You instantiate VertexAI with your Google Cloud project ID, and the region location of your choice. To pass images to Gemini, you should either pass the bytes directly, or you can pass a URI of an image stored in a cloud storage bucket (like gs://my-bucket/my-img.jpg). You create an instance of the model. Here, I&#39;m using gemini-pro-vision. But later on, a gemini-ultra-vision model will also be available. Let&#39;s ask the model to generate content with the generateContent() method, by passing both a text prompt, and also an image. The ContentMaker and PartMaker classes are helpers to further simplify the creation of more advanced prompts that mix different modalities. But you could also just pass a simple string as argument of the generateContent() method. The ResponseHandler utility will retrieve all the text of the answer of the model.Instead of getting the whole output once all the text is generated, you can also adopt a streaming approach:<pre>model.generateContentStream(&quot;Why is the sky blue?&quot;) .stream() .forEach(System.out::print);</pre>You can also iterate over the stream with a for loop:<pre>ResponseStream&lt;GenerateContentResponse&gt; responseStream = model.generateContentStream(&quot;Why is the sky blue?&quot;); for (GenerateContentResponse responsePart: responseStream) { System.out.print(ResponseHandler.getText(responsePart)); }</pre><h3>Let’s chat!</h3>Gemini is a multimodal model, and it’s actually both a text generation model, but also a chat model. So you can chat with Gemini, and ask a series of questions in context. There’s a handy ChatSession utility class which simplifies the handling of the conversation:<pre>try (VertexAI vertexAI = new VertexAI(projectId, location)) { GenerateContentResponse response; GenerativeModel model = new GenerativeModel(modelName, vertexAI); ChatSession chatSession = new ChatSession(model); response = chatSession.sendMessage(&quot;Hello.&quot;); System.out.println(ResponseHandler.getText(response)); response = chatSession.sendMessage(&quot;What are all the colors in a rainbow?&quot;); System.out.println(ResponseHandler.getText(response)); response = chatSession.sendMessage(&quot;Why does it appear when it rains?&quot;); System.out.println(ResponseHandler.getText(response)); }</pre>This is convenient to use ChatSession as it takes care of keeping track of past questions from the user, and answers from the assistant.<h3>Going further</h3>This is just a few examples of the capabilities of Gemini. Be sure to check out some of the <a href="https://github.com/GoogleCloudPlatform/java-docs-samples/tree/main/vertexai/snippets/src/main/java/vertexai/gemini">samples that are available on Github</a>. Read <a href="https://cloud.google.com/vertex-ai/docs/generative-ai/start/quickstarts/quickstart-multimodal?hl=en">more about Gemini and Generative AI</a> in the Google Cloud documentation.Originally published at <a href="https://glaforge.dev/posts/2023/12/13/get-started-with-gemini-in-java/">https://glaforge.dev</a> on December 13, 2023.<img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=923f2069ea4d" width="1" height="1" alt=""><hr><a href="https://medium.com/google-cloud/get-started-with-gemini-in-java-923f2069ea4d">Get Started with Gemini in Java</a> was originally published in <a href="https://medium.com/google-cloud">Google Cloud - Community</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.

Author

Link

Published date

Image url

Feed url

Guid

Hidden blurb

--- !ruby/object:Feedjira::Parser::RSSEntry
title: Get Started with Gemini in Java
url: https://medium.com/google-cloud/get-started-with-gemini-in-java-923f2069ea4d?source=rss-431147437aeb------2
author: Guillaume Laforge
categories:
- java
- gcp-app-dev
- generative-ai-tools
- google-cloud-platform
- machine-learning
published: 2023-12-13 00:00:25.000000000 Z
entry_id: !ruby/object:Feedjira::Parser::GloballyUniqueIdentifier
 is_perma_link: 'false'
 guid: https://medium.com/p/923f2069ea4d
carlessian_info:
 news_filer_version: 2
 newspaper: Guillaume Laforge - Medium
 macro_region: Blogs
rss_fields:
- title
- url
- author
- categories
- published
- entry_id
- content
content: '<figure><img alt="Logo of the Gemini large language model launched by Google"
 src="https://cdn-images-1.medium.com/max/1024/0*4ohfuLfaP-ZAbo5_" /></figure>Google
 announced today the availability of <a href="https://cloud.google.com/blog/products/ai-machine-learning/gemini-support-on-vertex-ai">Gemini</a>,
 its latest and more powerful Large Language Model. Gemini is multimodal,
 which means it’s able to consume not only text, but also images or videos.I
 had the pleasure of working on the Java samples and help with the Java SDK, with
 wonderful engineer colleagues, and I’d like to share some examples of what
 you can do with Gemini, using Java!First of all, you’ll need to
 have an account on Google Cloud and created a project. The Vertex AI API should
 be enabled, to be able to access the Generative AI services, and in particular the
 Gemini large language model. Be sure to check out the <a href="https://cloud.google.com/vertex-ai/docs/generative-ai/start/quickstarts/quickstart-multimodal?hl=en">instructions</a>.<h3>Preparing
 your project build</h3>To get started with some coding, you’ll need to create
 a Gradle or a Maven build file that requires the Google Cloud libraries BOM, and
 the google-cloud-vertexai library. Here&#39;s an example with Maven:<pre>... &lt;dependencyManagement&gt; &lt;dependencies&gt; &lt;dependency&gt; &lt;artifactId&gt;libraries-bom&lt;/artifactId&gt; &lt;groupId&gt;com.google.cloud&lt;/groupId&gt; &lt;scope&gt;import&lt;/scope&gt; &lt;type&gt;pom&lt;/type&gt; &lt;version&gt;26.29.0&lt;/version&gt; &lt;/dependency&gt; &lt;/dependencies&gt; &lt;/dependencyManagement&gt; &lt;dependencies&gt; &lt;dependency&gt; &lt;groupId&gt;com.google.cloud&lt;/groupId&gt; &lt;artifactId&gt;google-cloud-vertexai&lt;/artifactId&gt; &lt;/dependency&gt; ... &lt;/dependencies&gt; ...</pre><h3>Your
 first queries</h3>Now let’s have a look at our first multimodal example, mixing
 text prompts and images:<pre>try (VertexAI vertexAI = new VertexAI(projectId,
 location)) { byte[] imageBytes = Base64.getDecoder().decode(dataImageBase64); GenerativeModel
 model = new GenerativeModel(&quot;gemini-pro-vision&quot;, vertexAI); GenerateContentResponse
 response = model.generateContent( ContentMaker.fromMultiModalData( &quot;What
 is this image about?&quot;, PartMaker.fromMimeTypeAndData(&quot;image/jpg&quot;,
 imageBytes) )); System.out.println(ResponseHandler.getText(response)); }</pre>You
 instantiate VertexAI with your Google Cloud project ID, and the region location
 of your choice. To pass images to Gemini, you should either pass the bytes directly,
 or you can pass a URI of an image stored in a cloud storage bucket (like gs://my-bucket/my-img.jpg).
 You create an instance of the model. Here, I&#39;m using gemini-pro-vision. But
 later on, a gemini-ultra-vision model will also be available. Let&#39;s ask the
 model to generate content with the generateContent() method, by passing both a text
 prompt, and also an image. The ContentMaker and PartMaker classes are helpers to
 further simplify the creation of more advanced prompts that mix different modalities.
 But you could also just pass a simple string as argument of the generateContent()
 method. The ResponseHandler utility will retrieve all the text of the answer of
 the model.Instead of getting the whole output once all the text is generated,
 you can also adopt a streaming approach:<pre>model.generateContentStream(&quot;Why
 is the sky blue?&quot;) .stream() .forEach(System.out::print);</pre>You
 can also iterate over the stream with a for loop:<pre>ResponseStream&lt;GenerateContentResponse&gt;
 responseStream = model.generateContentStream(&quot;Why is the sky blue?&quot;); for
 (GenerateContentResponse responsePart: responseStream) { System.out.print(ResponseHandler.getText(responsePart)); }</pre><h3>Let’s
 chat!</h3>Gemini is a multimodal model, and it’s actually both a text generation
 model, but also a chat model. So you can chat with Gemini, and ask a series of questions
 in context. There’s a handy ChatSession utility class which simplifies the handling
 of the conversation:<pre>try (VertexAI vertexAI = new VertexAI(projectId, location))
 { GenerateContentResponse response; GenerativeModel model = new
 GenerativeModel(modelName, vertexAI); ChatSession chatSession = new ChatSession(model); response
 = chatSession.sendMessage(&quot;Hello.&quot;); System.out.println(ResponseHandler.getText(response)); response
 = chatSession.sendMessage(&quot;What are all the colors in a rainbow?&quot;); System.out.println(ResponseHandler.getText(response)); response
 = chatSession.sendMessage(&quot;Why does it appear when it rains?&quot;); System.out.println(ResponseHandler.getText(response)); }</pre>This
 is convenient to use ChatSession as it takes care of keeping track of past questions
 from the user, and answers from the assistant.<h3>Going further</h3>This
 is just a few examples of the capabilities of Gemini. Be sure to check out some
 of the <a href="https://github.com/GoogleCloudPlatform/java-docs-samples/tree/main/vertexai/snippets/src/main/java/vertexai/gemini">samples
 that are available on Github</a>. Read <a href="https://cloud.google.com/vertex-ai/docs/generative-ai/start/quickstarts/quickstart-multimodal?hl=en">more
 about Gemini and Generative AI</a> in the Google Cloud documentation.Originally
 published at <a href="https://glaforge.dev/posts/2023/12/13/get-started-with-gemini-in-java/">https://glaforge.dev</a>
 on December 13, 2023.<img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=923f2069ea4d"
 width="1" height="1" alt=""><hr><a href="https://medium.com/google-cloud/get-started-with-gemini-in-java-923f2069ea4d">Get
 Started with Gemini in Java</a> was originally published in <a href="https://medium.com/google-cloud">Google
 Cloud - Community</a> on Medium, where people are continuing the conversation by
 highlighting and responding to this story.'

Language

Active

Ricc internal notes

Ricc source

Show this article Back to articles