♊️ GemiNews 🗞️
(dev)
🏡
📰 Articles
🏷️ Tags
🧠 Queries
📈 Graphs
☁️ Stats
💁🏻 Assistant
💬
🎙️
Demo 1: Embeddings + Recommendation
Demo 2: Bella RAGa
Demo 3: NewRetriever
Demo 4: Assistant function calling
Editing article
Title
Summary
Content
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*4xCKDkXmAXkIXMbs.png" /></figure><h3>Abstract</h3><p>The Gemini API allows the generating of text from uploaded files using Google Apps Script. It expands the potential of various scripting languages for diverse applications.</p><h3>Introduction</h3><p>With the release of the LLM model Gemini as an API on Vertex AI and Google AI Studio, a world of possibilities has opened up. <a href="https://deepmind.google/technologies/gemini/#introduction">Ref</a> The Gemini API significantly expands the potential of various scripting languages and paves the way for diverse applications. Also, recently, Gemini 1.5 in AI Studio has been released. <a href="https://blog.google/technology/ai/google-gemini-next-generation-model-february-2024/#sundar-note">Ref</a> In the near future, Gemini 1.5 API will be also released soon.</p><p>Recently, the files got to be able to be uploaded with Gemini API. <a href="https://ai.google.dev/api/rest/v1beta/files">Ref1</a> and <a href="https://ai.google.dev/api/rest/v1beta/media">Ref2</a> When this is used, the text is generated using the uploaded files. This report introduces the sample scripts for uploading the files and generating the texts using Google Apps Script.</p><h3>Usage</h3><p>In order to test this script, please do the following steps.</p><h3>1. Create an API key</h3><p>Please access <a href="https://makersuite.google.com/app/apikey">https://makersuite.google.com/app/apikey</a> and create your API key. At that time, please enable Generative Language API at the API console. This API key is used for this sample script.</p><p>This official document can be also seen. <a href="https://ai.google.dev/">Ref</a>.</p><h3>2. Create a Google Apps Script project</h3><p>In this report, Google Apps Script is used. Of course, the method introducing this report can be also used in other languages.</p><p>Please create a standalone Google Apps Script project. Of course, this script can be also used with the container-bound script.</p><p>And, please open the script editor of the Google Apps Script project.</p><h3>3. Steps of script</h3><p>Here, it introduces the following 4 sample scripts.</p><ol><li>Upload a file with <a href="https://ai.google.dev/api/rest/v1beta/media">“Method: files.list”</a>.</li><li>Confirm the uploaded file with <a href="https://ai.google.dev/api/rest/v1beta/files">“Method: media.upload”</a>.</li><li>Generate content using the uploaded file with <a href="https://ai.google.dev/api/rest/v1beta/models/generateContent">“Method: models.generateContent”</a>.</li><li>Delete the uploaded file with <a href="https://ai.google.dev/api/rest/v1beta/files/delete">“Method: files.delete”</a>.</li></ol><p>Limitation of the uploaded file <a href="https://github.com/google-gemini/gemini-api-cookbook/blob/main/preview/file-api/File_API.ipynb">Ref</a></p><blockquote><em>Can only be used with model.generateContent or model.streamGenerateContent Automatic file deletion after 2 days Maximum 2GB per file, 20GB limit per project No downloads allowed</em></blockquote><h3>4. Script</h3><h3>Upload a file</h3><p>You can see the official document at <a href="https://ai.google.dev/api/rest/v1beta/media/upload">Method: media.upload</a>. The sample script of Google Apps Script is as follows.</p><p>Please set your API key and the file ID of the image file. Here, PNG image is used.</p><pre>function sample1() {<br> const apiKey = "###"; // Please set your API key.<br> const fileId = "###"; // Please set the file ID of the image file. Here, PNG image is used.<br><br> const url = `https://generativelanguage.googleapis.com/upload/v1beta/files?uploadType=multipart&key=${apiKey}`;<br> const metadata = {<br> file: { displayName: DriveApp.getFileById(fileId).getName() },<br> };<br> const payload = {<br> metadata: Utilities.newBlob(JSON.stringify(metadata), "application/json"),<br> file: UrlFetchApp.fetch(<br> `https://drive.google.com/thumbnail?sz=w1000&id=${fileId}`,<br> { headers: { authorization: "Bearer " + ScriptApp.getOAuthToken() } }<br> ).getBlob(),<br> };<br> const options = {<br> method: "post",<br> payload: payload,<br> muteHttpExceptions: true,<br> };<br> const res = UrlFetchApp.fetch(url, options).getContentText();<br> console.log(res);<br>}</pre><p>When this script is run, the following value is returned.</p><pre>{<br> "file": {<br> "name": "files/###",<br> "displayName": "###",<br> "mimeType": "image/jpeg",<br> "sizeBytes": "123456",<br> "createTime": "2024-03-30T01:23:00.000000Z",<br> "updateTime": "2024-03-30T01:23:00.000000Z",<br> "expirationTime": "2024-04-01T01:23:00.000000Z",<br> "sha256Hash": "###",<br> "uri": "https://generativelanguage.googleapis.com/v1beta/files/###"<br> }<br>}</pre><p>The values of mimeType and uri are used with generateContent.</p><p>In this sample, I used uploadType=multipart because of the small size of the image file. If you want to upload a large file, I think that resumable upload can be also used.</p><h3>Get the file list</h3><p>You can see the official document at <a href="https://ai.google.dev/api/rest/v1beta/files/list">Method: files.list</a>. The sample script of Google Apps Script is as follows.</p><p>Please set your API key.</p><pre>function sample2() {<br> const apiKey = "###"; // Please set your API key.<br><br> const url = `https://generativelanguage.googleapis.com/v1beta/files?pageSize=100&key=${apiKey}`;<br> const res = UrlFetchApp.fetch(url);<br> console.log(res.getContentText());<br>}</pre><p>When this script is run, the following value is returned.</p><pre>{<br> "files": [<br> {<br> "name": "files/###",<br> "displayName": "###",<br> "mimeType": "image/jpeg",<br> "sizeBytes": "123456",<br> "createTime": "2024-03-30T01:23:00.000000Z",<br> "updateTime": "2024-03-30T01:23:00.000000Z",<br> "expirationTime": "2024-04-01T01:23:00.000000Z",<br> "sha256Hash": "###",<br> "uri": "https://generativelanguage.googleapis.com/v1beta/files/###"<br> },<br> ,<br> ,<br> ,<br> ]<br>}</pre><p>When the number of files is more than 100, please retrieve all files using pageToken.</p><h3>Generate content from the uploaded file</h3><p>You can see the official document at <a href="https://ai.google.dev/api/rest/v1beta/models/generateContent">Method: models.generateContent</a>. The sample script of Google Apps Script is as follows.</p><p>Please set your API key, the URI of the uploaded file, and the mimeType of the file.</p><pre>function sample3() {<br> const apiKey = "###"; // Please set your API key.<br> const fileUri = "https://generativelanguage.googleapis.com/v1beta/files/###"; // Please set your file uri of the uploaded file.<br> const mimeType = "image/jpeg"; // Please set the mimeType of the uploaded file.<br><br> const q = "Describe the image and count apples in the image.";<br> const model = "models/gemini-1.5-pro-gf-fc";<br> const baseUrl = `https://generativelanguage.googleapis.com/v1beta/${model}`;<br> const payload = {<br> contents: [{ parts: [{ text: q }, { fileData: { fileUri, mimeType } }] }],<br> };<br> const options = {<br> payload: JSON.stringify(payload),<br> contentType: "application/json",<br> muteHttpExceptions: true,<br> };<br> const res = UrlFetchApp.fetch(<br> `${baseUrl}:generateContent?key=${apiKey}`,<br> options<br> );<br> console.log(res.getContentText());<br>}</pre><p>In this sample, the following image created by Gemini was uploaded as a sample file and was used.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*2K3xjxp7uwYgCx7-.png" /></figure><p>There are 12 apples including 7 red apples, 4 green apples, and 1 yellow apple are shown in the image. This image was generated by Gemini.</p><p>When this script is run, the following generated contents are returned.</p><ul><li>At the model models/gemini-1.0-pro-latest, Image input modality is not enabled for models/gemini-1.0-pro-latest was returned.</li><li>At the model models/gemini-1.0-pro-vision-latest, There are 10 apples in the image. Four red, five green, and one yellow. was returned.</li><li>At the model models/gemini-1.5-pro-latest, The image shows a group of apples on a wooden table. There are red, green, and yellow apples. There are 14 apples in total. was returned.</li><li>At the model models/gemini-1.5-pro-gf-fc, The image shows a group of apples on a wooden table. There are 13 apples in total. The apples are of different colors, including red, green, and yellow. The apples are arranged in a random pattern on the table. The light is coming from the left side of the image, and it is casting shadows on the apples and the table. was returned.</li></ul><h3>Delete the uploaded file</h3><p>You can see the official document at <a href="https://ai.google.dev/api/rest/v1beta/files/delete">Method: files.delete</a>. The sample script of Google Apps Script is as follows.</p><p>Please set your API key and the name of the uploaded file.</p><pre>function sample4() {<br> const apiKey = "###"; // Please set your API key.<br> const name = "files/###"; // Please set the name of the uploaded file.<br><br> const url = `https://generativelanguage.googleapis.com/v1beta/${name}?key=${apiKey}`;<br> const res = UrlFetchApp.fetch(url, { method: "delete" });<br> console.log(res.getContentText()); // {}<br>}</pre><p>In this case, an empty object like {} is returned.</p><p>In the current stage, the expiration time of the uploaded file is 2 days. So, the uploaded file is automatically deleted 2 days later.</p><h3>Summary</h3><p>In this report, we present sample scripts for using the Gemini API’s generateContent function with uploaded files. Our findings are as follows:</p><ul><li>Uploading files, retrieving file lists, and deleting files all functioned smoothly using an API key. Also, I heard that at Google APIs, both “snake_case” and “camelCase” within the request body. This was confirmed through testing.</li><li>For generating content from uploaded image files, Gemini 1.5 API models models/gemini-1.5-pro-latest and models/gemini-1.5-pro-gf-fc can be used for image analysis. However, accurately counting objects within the image might still be challenging.</li><li>Currently, uploading text, CSV, and PDF files results in an error message like “Request contains an invalid argument.” It appears that only image and movie files are supported at this stage. We anticipate this limitation to be addressed in a future update.</li></ul><h3>Note</h3><ul><li>The top illustration was created by <a href="https://gemini.google.com/">Gemini</a> with giving the abstract.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5777f1c902ab" width="1" height="1" alt=""><hr><p><a href="https://medium.com/google-cloud/generating-texts-using-files-uploaded-by-gemini-1-5-api-5777f1c902ab">Generating Texts using Files Uploaded by Gemini 1.5 API</a> was originally published in <a href="https://medium.com/google-cloud">Google Cloud - Community</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>
Author
Link
Published date
Image url
Feed url
Guid
Hidden blurb
--- !ruby/object:Feedjira::Parser::RSSEntry title: Generating Texts using Files Uploaded by Gemini 1.5 API url: https://medium.com/google-cloud/generating-texts-using-files-uploaded-by-gemini-1-5-api-5777f1c902ab?source=rss----e52cf94d98af---4 author: Kanshi Tanaike categories: - gemini - google-apps-script - generative-ai - google-cloud-platform - machine-learning published: 2024-03-31 02:50:43.000000000 Z entry_id: !ruby/object:Feedjira::Parser::GloballyUniqueIdentifier is_perma_link: 'false' guid: https://medium.com/p/5777f1c902ab carlessian_info: news_filer_version: 2 newspaper: Google Cloud - Medium macro_region: Blogs rss_fields: - title - url - author - categories - published - entry_id - content content: '<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*4xCKDkXmAXkIXMbs.png" /></figure><h3>Abstract</h3><p>The Gemini API allows the generating of text from uploaded files using Google Apps Script. It expands the potential of various scripting languages for diverse applications.</p><h3>Introduction</h3><p>With the release of the LLM model Gemini as an API on Vertex AI and Google AI Studio, a world of possibilities has opened up. <a href="https://deepmind.google/technologies/gemini/#introduction">Ref</a> The Gemini API significantly expands the potential of various scripting languages and paves the way for diverse applications. Also, recently, Gemini 1.5 in AI Studio has been released. <a href="https://blog.google/technology/ai/google-gemini-next-generation-model-february-2024/#sundar-note">Ref</a> In the near future, Gemini 1.5 API will be also released soon.</p><p>Recently, the files got to be able to be uploaded with Gemini API. <a href="https://ai.google.dev/api/rest/v1beta/files">Ref1</a> and <a href="https://ai.google.dev/api/rest/v1beta/media">Ref2</a> When this is used, the text is generated using the uploaded files. This report introduces the sample scripts for uploading the files and generating the texts using Google Apps Script.</p><h3>Usage</h3><p>In order to test this script, please do the following steps.</p><h3>1. Create an API key</h3><p>Please access <a href="https://makersuite.google.com/app/apikey">https://makersuite.google.com/app/apikey</a> and create your API key. At that time, please enable Generative Language API at the API console. This API key is used for this sample script.</p><p>This official document can be also seen. <a href="https://ai.google.dev/">Ref</a>.</p><h3>2. Create a Google Apps Script project</h3><p>In this report, Google Apps Script is used. Of course, the method introducing this report can be also used in other languages.</p><p>Please create a standalone Google Apps Script project. Of course, this script can be also used with the container-bound script.</p><p>And, please open the script editor of the Google Apps Script project.</p><h3>3. Steps of script</h3><p>Here, it introduces the following 4 sample scripts.</p><ol><li>Upload a file with <a href="https://ai.google.dev/api/rest/v1beta/media">“Method: files.list”</a>.</li><li>Confirm the uploaded file with <a href="https://ai.google.dev/api/rest/v1beta/files">“Method: media.upload”</a>.</li><li>Generate content using the uploaded file with <a href="https://ai.google.dev/api/rest/v1beta/models/generateContent">“Method: models.generateContent”</a>.</li><li>Delete the uploaded file with <a href="https://ai.google.dev/api/rest/v1beta/files/delete">“Method: files.delete”</a>.</li></ol><p>Limitation of the uploaded file <a href="https://github.com/google-gemini/gemini-api-cookbook/blob/main/preview/file-api/File_API.ipynb">Ref</a></p><blockquote><em>Can only be used with model.generateContent or model.streamGenerateContent Automatic file deletion after 2 days Maximum 2GB per file, 20GB limit per project No downloads allowed</em></blockquote><h3>4. Script</h3><h3>Upload a file</h3><p>You can see the official document at <a href="https://ai.google.dev/api/rest/v1beta/media/upload">Method: media.upload</a>. The sample script of Google Apps Script is as follows.</p><p>Please set your API key and the file ID of the image file. Here, PNG image is used.</p><pre>function sample1() {<br> const apiKey = "###"; // Please set your API key.<br> const fileId = "###"; // Please set the file ID of the image file. Here, PNG image is used.<br><br> const url = `https://generativelanguage.googleapis.com/upload/v1beta/files?uploadType=multipart&key=${apiKey}`;<br> const metadata = {<br> file: { displayName: DriveApp.getFileById(fileId).getName() },<br> };<br> const payload = {<br> metadata: Utilities.newBlob(JSON.stringify(metadata), "application/json"),<br> file: UrlFetchApp.fetch(<br> `https://drive.google.com/thumbnail?sz=w1000&id=${fileId}`,<br> { headers: { authorization: "Bearer " + ScriptApp.getOAuthToken() } }<br> ).getBlob(),<br> };<br> const options = {<br> method: "post",<br> payload: payload,<br> muteHttpExceptions: true,<br> };<br> const res = UrlFetchApp.fetch(url, options).getContentText();<br> console.log(res);<br>}</pre><p>When this script is run, the following value is returned.</p><pre>{<br> "file": {<br> "name": "files/###",<br> "displayName": "###",<br> "mimeType": "image/jpeg",<br> "sizeBytes": "123456",<br> "createTime": "2024-03-30T01:23:00.000000Z",<br> "updateTime": "2024-03-30T01:23:00.000000Z",<br> "expirationTime": "2024-04-01T01:23:00.000000Z",<br> "sha256Hash": "###",<br> "uri": "https://generativelanguage.googleapis.com/v1beta/files/###"<br> }<br>}</pre><p>The values of mimeType and uri are used with generateContent.</p><p>In this sample, I used uploadType=multipart because of the small size of the image file. If you want to upload a large file, I think that resumable upload can be also used.</p><h3>Get the file list</h3><p>You can see the official document at <a href="https://ai.google.dev/api/rest/v1beta/files/list">Method: files.list</a>. The sample script of Google Apps Script is as follows.</p><p>Please set your API key.</p><pre>function sample2() {<br> const apiKey = "###"; // Please set your API key.<br><br> const url = `https://generativelanguage.googleapis.com/v1beta/files?pageSize=100&key=${apiKey}`;<br> const res = UrlFetchApp.fetch(url);<br> console.log(res.getContentText());<br>}</pre><p>When this script is run, the following value is returned.</p><pre>{<br> "files": [<br> {<br> "name": "files/###",<br> "displayName": "###",<br> "mimeType": "image/jpeg",<br> "sizeBytes": "123456",<br> "createTime": "2024-03-30T01:23:00.000000Z",<br> "updateTime": "2024-03-30T01:23:00.000000Z",<br> "expirationTime": "2024-04-01T01:23:00.000000Z",<br> "sha256Hash": "###",<br> "uri": "https://generativelanguage.googleapis.com/v1beta/files/###"<br> },<br> ,<br> ,<br> ,<br> ]<br>}</pre><p>When the number of files is more than 100, please retrieve all files using pageToken.</p><h3>Generate content from the uploaded file</h3><p>You can see the official document at <a href="https://ai.google.dev/api/rest/v1beta/models/generateContent">Method: models.generateContent</a>. The sample script of Google Apps Script is as follows.</p><p>Please set your API key, the URI of the uploaded file, and the mimeType of the file.</p><pre>function sample3() {<br> const apiKey = "###"; // Please set your API key.<br> const fileUri = "https://generativelanguage.googleapis.com/v1beta/files/###"; // Please set your file uri of the uploaded file.<br> const mimeType = "image/jpeg"; // Please set the mimeType of the uploaded file.<br><br> const q = "Describe the image and count apples in the image.";<br> const model = "models/gemini-1.5-pro-gf-fc";<br> const baseUrl = `https://generativelanguage.googleapis.com/v1beta/${model}`;<br> const payload = {<br> contents: [{ parts: [{ text: q }, { fileData: { fileUri, mimeType } }] }],<br> };<br> const options = {<br> payload: JSON.stringify(payload),<br> contentType: "application/json",<br> muteHttpExceptions: true,<br> };<br> const res = UrlFetchApp.fetch(<br> `${baseUrl}:generateContent?key=${apiKey}`,<br> options<br> );<br> console.log(res.getContentText());<br>}</pre><p>In this sample, the following image created by Gemini was uploaded as a sample file and was used.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*2K3xjxp7uwYgCx7-.png" /></figure><p>There are 12 apples including 7 red apples, 4 green apples, and 1 yellow apple are shown in the image. This image was generated by Gemini.</p><p>When this script is run, the following generated contents are returned.</p><ul><li>At the model models/gemini-1.0-pro-latest, Image input modality is not enabled for models/gemini-1.0-pro-latest was returned.</li><li>At the model models/gemini-1.0-pro-vision-latest, There are 10 apples in the image. Four red, five green, and one yellow. was returned.</li><li>At the model models/gemini-1.5-pro-latest, The image shows a group of apples on a wooden table. There are red, green, and yellow apples. There are 14 apples in total. was returned.</li><li>At the model models/gemini-1.5-pro-gf-fc, The image shows a group of apples on a wooden table. There are 13 apples in total. The apples are of different colors, including red, green, and yellow. The apples are arranged in a random pattern on the table. The light is coming from the left side of the image, and it is casting shadows on the apples and the table. was returned.</li></ul><h3>Delete the uploaded file</h3><p>You can see the official document at <a href="https://ai.google.dev/api/rest/v1beta/files/delete">Method: files.delete</a>. The sample script of Google Apps Script is as follows.</p><p>Please set your API key and the name of the uploaded file.</p><pre>function sample4() {<br> const apiKey = "###"; // Please set your API key.<br> const name = "files/###"; // Please set the name of the uploaded file.<br><br> const url = `https://generativelanguage.googleapis.com/v1beta/${name}?key=${apiKey}`;<br> const res = UrlFetchApp.fetch(url, { method: "delete" });<br> console.log(res.getContentText()); // {}<br>}</pre><p>In this case, an empty object like {} is returned.</p><p>In the current stage, the expiration time of the uploaded file is 2 days. So, the uploaded file is automatically deleted 2 days later.</p><h3>Summary</h3><p>In this report, we present sample scripts for using the Gemini API’s generateContent function with uploaded files. Our findings are as follows:</p><ul><li>Uploading files, retrieving file lists, and deleting files all functioned smoothly using an API key. Also, I heard that at Google APIs, both “snake_case” and “camelCase” within the request body. This was confirmed through testing.</li><li>For generating content from uploaded image files, Gemini 1.5 API models models/gemini-1.5-pro-latest and models/gemini-1.5-pro-gf-fc can be used for image analysis. However, accurately counting objects within the image might still be challenging.</li><li>Currently, uploading text, CSV, and PDF files results in an error message like “Request contains an invalid argument.” It appears that only image and movie files are supported at this stage. We anticipate this limitation to be addressed in a future update.</li></ul><h3>Note</h3><ul><li>The top illustration was created by <a href="https://gemini.google.com/">Gemini</a> with giving the abstract.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5777f1c902ab" width="1" height="1" alt=""><hr><p><a href="https://medium.com/google-cloud/generating-texts-using-files-uploaded-by-gemini-1-5-api-5777f1c902ab">Generating Texts using Files Uploaded by Gemini 1.5 API</a> was originally published in <a href="https://medium.com/google-cloud">Google Cloud - Community</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>'
Language
Active
Ricc internal notes
Imported via /Users/ricc/git/gemini-news-crawler/webapp/db/seeds.d/import-feedjira.rb on 2024-03-31 22:53:45 +0200. Content is EMPTY here. Entried: title,url,author,categories,published,entry_id,content. TODO add Newspaper: filename = /Users/ricc/git/gemini-news-crawler/webapp/db/seeds.d/../../../crawler/out/feedjira/Blogs/Google Cloud - Medium/2024-03-31-Generating_Texts_using_Files_Uploaded_by_Gemini_1.5_API-v2.yaml
Ricc source
Show this article
Back to articles