♊️ GemiNews 🗞️ (dev)

Demo 1: Embeddings + Recommendation Demo 2: Bella RAGa Demo 3: NewRetriever Demo 4: Assistant function calling

🗞️Introducing Cloud Run volume mounts: connect your app to Cloud Storage or NFS

🗿Semantically Similar Articles (by :title_embedding)

Introducing Cloud Run volume mounts: connect your app to Cloud Storage or NFS

Article Image
2024-03-22 - Christoph Stanger (from Google Cloud Blog)

We’ve built Cloud Run, a fully managed container platform, directly on top of Google’s scalable infrastructure to simplify developers’ lives and make it easier to build cloud-native applications.As it stands, each Cloud Run instance has access to its own local file system. But what if you have an existing application that expects to access shared data stored in a local file system? Without a straightforward way to mount storage systems like file servers or Cloud Storage buckets, developers had to use complex solutions or look to other services. Today, we’re excited to launch a new feature in preview: volume mounts.With volume mounts, mounting a volume in a Cloud Run service or job is a single command. You can mount a Cloud Storage bucket or an NFS share, like a Cloud Filestore instance. This allows your container to seamlessly access the storage bucket or file server content as if the files were local, utilizing file system semantics for a familiar experience.You can mount a Cloud Storage bucket by updating your Cloud Run service with the following command (you can find more details and instructions in the Try it out section below): code_block <ListValue: [StructValue([('code', 'gcloud beta run services update [SERVICE_NAME] \\\r\n--execution-environment gen2 \\ \r\n--add-volume=name=v_mount,type=cloud-storage,bucket=[YOUR_BUCKET_NAME] \\\r\n--add-volume-mount=volume=v_mount,mount-path=[MOUNT_PATH]'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x3ecdc41c60a0>)])]> Volume mounts come in handy in a number of situations.  1. Store app configuration Let's consider a scenario where you need to add config files to your service. When applications launch, they often need to gather information about their environment and load initial settings to determine their behavior. In the past, we’ve seen customers use Secret Manager to store and mount this information, but for configuration data that doesn’t need to be kept secret, Cloud Storage is a more straightforward solution. Simply put all your configuration into a file in your preferred format, upload the file to a Cloud Storage bucket, and mount the bucket in your Cloud Run service or job at the required path.  Since Cloud Run supports pulling public container images from Docker Hub directly, mounting your own config files to customize official images (like Grafana or Nginx) becomes very convenient. There's no need to build, add your config files and host your own container image. Just deploy an official container image from Docker Hub directly, store your config files in a Cloud Storage bucket and mount them where they are expected. 2. Event-driven Cloud Storage handlers Many applications are built using an event-driven design pattern. A common use case is executing custom code based on a new file being uploaded to a Cloud Storage bucket. EventArc is a great tool to listen to such events and to trigger a Cloud Run service directly. It forwards all relevant event metadata, including the file name and location — but not the file itself. Until now, to retrieve and process the file, you needed to use the Cloud Storage client SDK to explicitly retrieve it.  With volume mounts you can now mount the relevant bucket directly. This allows you to access the file directly via the filesystem, eliminating the need for custom code to fetch it. 3. Load a vector database file If you have your Langchain application deployed on Cloud Run, you may need a vector database, like ChromaDB. Because the indexed documents are constantly changing, mounting an NFS storage is a great way to keep your service stateless and externalize your ChromaDB collection from the container — all while having a dedicated ingestion pipeline for new documents outside your service.  Persisted ChromaDB collections can grow large quickly, so Cloud Filestore is a fast option to access them from all your Cloud Run application instances. 4. Serve a static website For simple public-facing file hosting, you could directly use Cloud Storage’s static website hosting feature. However, if you need private networking features or a simple login experience via Identity-Aware Proxy (IAP), Cloud Run is an excellent choice.  Previously, you had to copy all the static files into your container image to host and serve them from there. However, that required you to rebuild the image and then redeploy your service every time you changed the static content. With volume mounts, you can now use a standard NGINX web server and serve your files from a mounted Cloud Storage bucket. Cloud Run accesses your files using standard file system semantics, so you can use the official, publicly hosted NGINX container image from DockerHub directly. You can now modify your static assets or add new ones as needed, with all changes taking effect for your Cloud Run service promptly and without downtime.  This setup provides access to all the valuable ingress features, including IAP, while retaining the flexibility of storing your files in a Cloud Storage bucket. To run a simple web server that serves files from your Cloud Storage bucket, deploy the official NGINX image from Docker Hub and mount the bucket to the directory where NGINX expects to find static content: '/usr/share/nginx/html'. You can do this with a single command: code_block <ListValue: [StructValue([('code', "gcloud beta run deploy --image nginx [SERVICE_NAME] \\\r\n--execution-environment gen2 --port 80 \\\r\n--add-volume=name=html-volume,type=cloud-storage,bucket=[YOUR_BUCKET_NAME],readonly=true \\\r\n--add-volume-mount=volume=html-volume,mount-path='/usr/share/nginx/html'"), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x3ecdc41c65b0>)])]> As a last step, be sure to set up content caching, either through NGINX content caching or Cloud CDN. Without caching, each request triggers a Cloud Storage get request, which can lead to increased costs as well as unnecessary latency for your users. Try it out You can mount a Cloud Storage bucket or any NFS file share by using a gcloud command, updating the Cloud Run YAML resource definition or by deploying via Terraform. For example, you can perform a source-based deployment to a new Cloud Run job and mount a Cloud Storage bucket with the following command: code_block <ListValue: [StructValue([('code', 'gcloud beta run jobs deploy [SERVICE_NAME] --source \\\r\n--execution-environment gen2 \\\r\n--add-volume=name=[VOLUME_NAME],type=cloud-storage,bucket=[BUCKET_NAME] \\ \r\n--add-volume-mount=volume=[VOLUME_NAME],mount-path=[MOUNT_PATH]'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x3ecdc41c6820>)])]> Similarly, you can mount any NFS file share as a volume in Cloud Run. If you don’t already have an NFS server, we recommend using Cloud Filestore, Google Cloud’s fully managed NFS offering.  For more information and to get started, take a look at our documentation: Mount a Cloud Storage bucket in a service or job Mount an NFS volume in a service or job We’re excited about how easy volume mounts in Cloud Run make it to access data, port existing applications, and even to configure some pre-built container images. Try this feature in preview today.

empty

[Technology] 🌎 https://cloud.google.com/blog/products/serverless/introducing-cloud-run-volume-mounts/ [🧠] [v2] article_embedding_description: {:llm_project_id=>"Unavailable", :llm_dimensions=>nil, :article_size=>15330, :llm_embeddings_model_name=>"textembedding-gecko"}
[🧠] [v1/3] title_embedding_description: {:ricc_notes=>"[embed-v3] Fixed on 9oct24. Only seems incompatible at first glance with embed v1.", :llm_project_id=>"unavailable possibly not using Vertex", :llm_dimensions=>nil, :article_size=>15330, :poly_field=>"title", :llm_embeddings_model_name=>"textembedding-gecko"}
[🧠] [v1/3] summary_embedding_description: {:ricc_notes=>"[embed-v3] Fixed on 9oct24. Only seems incompatible at first glance with embed v1.", :llm_project_id=>"unavailable possibly not using Vertex", :llm_dimensions=>nil, :article_size=>15330, :poly_field=>"summary", :llm_embeddings_model_name=>"textembedding-gecko"}
[🧠] As per bug https://github.com/palladius/gemini-news-crawler/issues/4 we can state this article belongs to titile/summary version: v3 (very few articles updated on 9oct24)

🗿article.to_s

------------------------------
Title: Introducing Cloud Run volume mounts: connect your app to Cloud Storage or NFS
Summary: We’ve built Cloud Run, a fully managed container platform, directly on top of Google’s scalable infrastructure to simplify developers’ lives and make it easier to build cloud-native applications.As it stands, each Cloud Run instance has access to its own local file system. But what if you have an existing application that expects to access shared data stored in a local file system? Without a straightforward way to mount storage systems like file servers or Cloud Storage buckets, developers had to use complex solutions or look to other services. Today, we’re excited to launch a new feature in preview: volume mounts.With volume mounts, mounting a volume in a Cloud Run service or job is a single command. You can mount a Cloud Storage bucket or an NFS share, like a Cloud Filestore instance. This allows your container to seamlessly access the storage bucket or file server content as if the files were local, utilizing file system semantics for a familiar experience.You can mount a Cloud Storage bucket by updating your Cloud Run service with the following command (you can find more details and instructions in the Try it out section below):

    code_block
    <ListValue: [StructValue([('code', 'gcloud beta run services update [SERVICE_NAME] \\\r\n--execution-environment gen2 \\ \r\n--add-volume=name=v_mount,type=cloud-storage,bucket=[YOUR_BUCKET_NAME]  \\\r\n--add-volume-mount=volume=v_mount,mount-path=[MOUNT_PATH]'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x3ecdc41c60a0>)])]>

Volume mounts come in handy in a number of situations. 
1. Store app configuration
Let's consider a scenario where you need to add config files to your service. When applications launch, they often need to gather information about their environment and load initial settings to determine their behavior.
In the past, we’ve seen customers use Secret Manager to store and mount this information, but for configuration data that doesn’t need to be kept secret, Cloud Storage is a more straightforward solution. Simply put all your configuration into a file in your preferred format, upload the file to a Cloud Storage bucket, and mount the bucket in your Cloud Run service or job at the required path. 
Since Cloud Run supports pulling public container images from Docker Hub directly, mounting your own config files to customize official images (like Grafana or Nginx) becomes very convenient. There's no need to build, add your config files and host your own container image. Just deploy an official container image from Docker Hub directly, store your config files in a Cloud Storage bucket and mount them where they are expected.







  
    
      
  

    

      
      
        
        
        
        
      
    

  
      
    
  





2. Event-driven Cloud Storage handlers
Many applications are built using an event-driven design pattern. A common use case is executing custom code based on a new file being uploaded to a Cloud Storage bucket. EventArc is a great tool to listen to such events and to trigger a Cloud Run service directly. It forwards all relevant event metadata, including the file name and location — but not the file itself. Until now, to retrieve and process the file, you needed to use the Cloud Storage client SDK to explicitly retrieve it. 
With volume mounts you can now mount the relevant bucket directly. This allows you to access the file directly via the filesystem, eliminating the need for custom code to fetch it.







  
    
      
  

    

      
      
        
        
        
        
      
    

  
      
    
  





3. Load a vector database file
If you have your Langchain application deployed on Cloud Run, you may need a vector database, like ChromaDB. Because the indexed documents are constantly changing, mounting an NFS storage is a great way to keep your service stateless and externalize your ChromaDB collection from the container — all while having a dedicated ingestion pipeline for new documents outside your service. 
Persisted ChromaDB collections can grow large quickly, so Cloud Filestore is a fast option to access them from all your Cloud Run application instances.







  
    
      
  

    

      
      
        
        
        
        
      
    

  
      
    
  





4. Serve a static website
For simple public-facing file hosting, you could directly use Cloud Storage’s static website hosting feature. However, if you need private networking features or a simple login experience via Identity-Aware Proxy (IAP), Cloud Run is an excellent choice. 
Previously, you had to copy all the static files into your container image to host and serve them from there. However, that required you to rebuild the image and then redeploy your service every time you changed the static content. 
With volume mounts, you can now use a standard NGINX web server and serve your files from a mounted Cloud Storage bucket. Cloud Run accesses your files using standard file system semantics, so you can use the official, publicly hosted NGINX container image from DockerHub directly. You can now modify your static assets or add new ones as needed, with all changes taking effect for your Cloud Run service promptly and without downtime. 
This setup provides access to all the valuable ingress features, including IAP, while retaining the flexibility of storing your files in a Cloud Storage bucket.







  
    
      
  

    

      
      
        
        
        
        
      
    

  
      
    
  





To run a simple web server that serves files from your Cloud Storage bucket, deploy the official NGINX image from Docker Hub and mount the bucket to the directory where NGINX expects to find static content: '/usr/share/nginx/html'. You can do this with a single command: 

    code_block
    <ListValue: [StructValue([('code', "gcloud beta run deploy --image nginx [SERVICE_NAME] \\\r\n--execution-environment gen2 --port 80 \\\r\n--add-volume=name=html-volume,type=cloud-storage,bucket=[YOUR_BUCKET_NAME],readonly=true \\\r\n--add-volume-mount=volume=html-volume,mount-path='/usr/share/nginx/html'"), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x3ecdc41c65b0>)])]>

As a last step, be sure to set up content caching, either through NGINX content caching or Cloud CDN. Without caching, each request triggers a Cloud Storage get request, which can lead to increased costs as well as unnecessary latency for your users.
Try it out
You can mount a Cloud Storage bucket or any NFS file share by using a gcloud command, updating the Cloud Run YAML resource definition or by deploying via Terraform. For example, you can perform a source-based deployment to a new Cloud Run job and mount a Cloud Storage bucket with the following command:

    code_block
    <ListValue: [StructValue([('code', 'gcloud beta run jobs deploy [SERVICE_NAME] --source \\\r\n--execution-environment gen2 \\\r\n--add-volume=name=[VOLUME_NAME],type=cloud-storage,bucket=[BUCKET_NAME] \\ \r\n--add-volume-mount=volume=[VOLUME_NAME],mount-path=[MOUNT_PATH]'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x3ecdc41c6820>)])]>

Similarly, you can mount any NFS file share as a volume in Cloud Run. If you don’t already have an NFS server, we recommend using Cloud Filestore, Google Cloud’s fully managed NFS offering. 
For more information and to get started, take a look at our documentation:


Mount a Cloud Storage bucket in a service or job


Mount an NFS volume in a service or job


We’re excited about how easy volume mounts in Cloud Run make it to access data, port existing applications, and even to configure some pre-built container images. Try this feature in preview today.

[content]
empty
[/content]

Author: Christoph Stanger
PublishedDate: 2024-03-22
Category: Technology
NewsPaper: Google Cloud Blog
Tags: Storage & Data Transfer, Application Modernization, Serverless
{"id"=>1345,
"title"=>"Introducing Cloud Run volume mounts: connect your app to Cloud Storage or NFS",
"summary"=>"

We’ve built Cloud Run, a fully managed container platform, directly on top of Google’s scalable infrastructure to simplify developers’ lives and make it easier to build cloud-native applications.

As it stands, each Cloud Run instance has access to its own local file system. But what if you have an existing application that expects to access shared data stored in a local file system? Without a straightforward way to mount storage systems like file servers or Cloud Storage buckets, developers had to use complex solutions or look to other services. Today, we’re excited to launch a new feature in preview: volume mounts.

With volume mounts, mounting a volume in a Cloud Run service or job is a single command. You can mount a Cloud Storage bucket or an NFS share, like a Cloud Filestore instance. This allows your container to seamlessly access the storage bucket or file server content as if the files were local, utilizing file system semantics for a familiar experience.

You can mount a Cloud Storage bucket by updating your Cloud Run service with the following command (you can find more details and instructions in the Try it out section below):

\n
\n
code_block
\n
<ListValue: [StructValue([('code', 'gcloud beta run services update [SERVICE_NAME] \\\\\\r\\n--execution-environment gen2 \\\\ \\r\\n--add-volume=name=v_mount,type=cloud-storage,bucket=[YOUR_BUCKET_NAME] \\\\\\r\\n--add-volume-mount=volume=v_mount,mount-path=[MOUNT_PATH]'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x3ecdc41c60a0>)])]>
\n
\n

Volume mounts come in handy in a number of situations. 

\n

1. Store app configuration

\n

Let's consider a scenario where you need to add config files to your service. When applications launch, they often need to gather information about their environment and load initial settings to determine their behavior.

\n

In the past, we’ve seen customers use Secret Manager to store and mount this information, but for configuration data that doesn’t need to be kept secret, Cloud Storage is a more straightforward solution. Simply put all your configuration into a file in your preferred format, upload the file to a Cloud Storage bucket, and mount the bucket in your Cloud Run service or job at the required path. 

\n

Since Cloud Run supports pulling public container images from Docker Hub directly, mounting your own config files to customize official images (like Grafana or Nginx) becomes very convenient. There's no need to build, add your config files and host your own container image. Just deploy an official container image from Docker Hub directly, store your config files in a Cloud Storage bucket and mount them where they are expected.

\n
\n\n\n\n\n\n\n \n
\n
\n \n\n
\n\n \n \n \n \n \n \n \n
\n\n \n
\n
\n \n\n\n\n\n
\n

2. Event-driven Cloud Storage handlers

\n

Many applications are built using an event-driven design pattern. A common use case is executing custom code based on a new file being uploaded to a Cloud Storage bucket. EventArc is a great tool to listen to such events and to trigger a Cloud Run service directly. It forwards all relevant event metadata, including the file name and location — but not the file itself. Until now, to retrieve and process the file, you needed to use the Cloud Storage client SDK to explicitly retrieve it. 

\n

With volume mounts you can now mount the relevant bucket directly. This allows you to access the file directly via the filesystem, eliminating the need for custom code to fetch it.

\n
\n\n\n\n\n\n\n \n
\n
\n \n\n
\n\n \n \n \n \n \n \n \n
\n\n \n
\n
\n \n\n\n\n\n
\n

3. Load a vector database file

\n

If you have your Langchain application deployed on Cloud Run, you may need a vector database, like ChromaDB. Because the indexed documents are constantly changing, mounting an NFS storage is a great way to keep your service stateless and externalize your ChromaDB collection from the container — all while having a dedicated ingestion pipeline for new documents outside your service. 

\n

Persisted ChromaDB collections can grow large quickly, so Cloud Filestore is a fast option to access them from all your Cloud Run application instances.

\n
\n\n\n\n\n\n\n \n
\n
\n \n\n
\n\n \n \n \n \n \n \n \n
\n\n \n
\n
\n \n\n\n\n\n
\n

4. Serve a static website

\n

For simple public-facing file hosting, you could directly use Cloud Storage’s static website hosting feature. However, if you need private networking features or a simple login experience via Identity-Aware Proxy (IAP), Cloud Run is an excellent choice. 

\n

Previously, you had to copy all the static files into your container image to host and serve them from there. However, that required you to rebuild the image and then redeploy your service every time you changed the static content.

\n

With volume mounts, you can now use a standard NGINX web server and serve your files from a mounted Cloud Storage bucket. Cloud Run accesses your files using standard file system semantics, so you can use the official, publicly hosted NGINX container image from DockerHub directly. You can now modify your static assets or add new ones as needed, with all changes taking effect for your Cloud Run service promptly and without downtime. 

\n

This setup provides access to all the valuable ingress features, including IAP, while retaining the flexibility of storing your files in a Cloud Storage bucket.

\n
\n\n\n\n\n\n\n \n
\n
\n \n\n
\n\n \n \n \n \n \n \n \n
\n\n \n
\n
\n \n\n\n\n\n
\n

To run a simple web server that serves files from your Cloud Storage bucket, deploy the official NGINX image from Docker Hub and mount the bucket to the directory where NGINX expects to find static content: '/usr/share/nginx/html'. You can do this with a single command:

\n
\n
code_block
\n
<ListValue: [StructValue([('code', "gcloud beta run deploy --image nginx [SERVICE_NAME] \\\\\\r\\n--execution-environment gen2 --port 80 \\\\\\r\\n--add-volume=name=html-volume,type=cloud-storage,bucket=[YOUR_BUCKET_NAME],readonly=true \\\\\\r\\n--add-volume-mount=volume=html-volume,mount-path='/usr/share/nginx/html'"), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x3ecdc41c65b0>)])]>
\n
\n

As a last step, be sure to set up content caching, either through NGINX content caching or Cloud CDN. Without caching, each request triggers a Cloud Storage get request, which can lead to increased costs as well as unnecessary latency for your users.

\n

Try it out

\n

You can mount a Cloud Storage bucket or any NFS file share by using a gcloud command, updating the Cloud Run YAML resource definition or by deploying via Terraform. For example, you can perform a source-based deployment to a new Cloud Run job and mount a Cloud Storage bucket with the following command:

\n
\n
code_block
\n
<ListValue: [StructValue([('code', 'gcloud beta run jobs deploy [SERVICE_NAME] --source \\\\\\r\\n--execution-environment gen2 \\\\\\r\\n--add-volume=name=[VOLUME_NAME],type=cloud-storage,bucket=[BUCKET_NAME] \\\\ \\r\\n--add-volume-mount=volume=[VOLUME_NAME],mount-path=[MOUNT_PATH]'), ('language', ''), ('caption', <wagtail.rich_text.RichText object at 0x3ecdc41c6820>)])]>
\n
\n

Similarly, you can mount any NFS file share as a volume in Cloud Run. If you don’t already have an NFS server, we recommend using Cloud Filestore, Google Cloud’s fully managed NFS offering. 

\n

For more information and to get started, take a look at our documentation:

\n
    \n
  • \n

    Mount a Cloud Storage bucket in a service or job

    \n
  • \n
  • \n

    Mount an NFS volume in a service or job

    \n
  • \n
\n

We’re excited about how easy volume mounts in Cloud Run make it to access data, port existing applications, and even to configure some pre-built container images. Try this feature in preview today.

",
"content"=>"",
"author"=>"Christoph Stanger",
"link"=>"https://cloud.google.com/blog/products/serverless/introducing-cloud-run-volume-mounts/",
"published_date"=>Fri, 22 Mar 2024 16:00:00.000000000 UTC +00:00,
"image_url"=>"https://storage.googleapis.com/gweb-cloudblog-publish/images/multicloud_2022_uvaqaSe.max-600x600.jpg",
"feed_url"=>"https://cloud.google.com/blog/products/serverless/introducing-cloud-run-volume-mounts/",
"language"=>nil,
"active"=>true,
"ricc_source"=>"feedjira::v1",
"created_at"=>Sun, 31 Mar 2024 21:23:53.247034000 UTC +00:00,
"updated_at"=>Mon, 21 Oct 2024 16:58:42.559079000 UTC +00:00,
"newspaper"=>"Google Cloud Blog",
"macro_region"=>"Technology"}
Edit this article
Back to articles