Editing article

Title

Summary

Content

<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G_gcJC7qUPp_bUfFU90ECA.png" /></figure><blockquote>Introduction</blockquote>Managing and monitoring your Google Cloud Platform (GCP) resources is crucial for maintaining optimal performance and identifying potential issues. While GCP offers its own monitoring tools, integrating them with a powerful visualization platform like Grafana can take your monitoring game to the next level. This blog will guide you through setting up a self-hosted Grafana server on a GCP VM and leveraging it to monitor your cloud resources.<blockquote>What is Grafana?</blockquote>Grafana is an open-source platform that allows you to create interactive dashboards to visualize metrics, logs, and traces from various data sources. It excels at transforming complex data sets into easily digestible charts, graphs, and other visual elements, enabling you to gain deeper insights into your system’s health and performance.<figure><img alt="" src="https://cdn-images-1.medium.com/max/456/1*BQpgEIUMeRnsbSmX106A0Q.png" /></figure><blockquote>Grafana OSS vs Grafana Cloud</blockquote>Grafana comes in two flavors: Open Source (OSS) and Cloud. The OSS version offers complete control and customization but requires self-hosting and maintenance. Grafana Cloud provides a managed service with features like automatic scaling and enterprise-grade security, ideal for those seeking a hassle-free experience. You can find more details on Grafana Cloud vs OSS version here : <a href="https://grafana.com/oss-vs-cloud/">https://grafana.com/oss-vs-cloud/</a>In this article we are focusing Grafana OSS version i.e self hosted Grafana.<blockquote>Creating Your Own Grafana Server on a GCP VM</blockquote>Provision a GCP VM: Choose a suitable VM instance type based on your expected load.I created basic debian VM.Install Grafana: Follow the official Grafana installation guide for your chosen operating system (e.g., Ubuntu, Debian). You can find download instructions here : <a href="https://grafana.com/grafana/download/10.4.0?edition=oss">https://grafana.com/grafana/download/10.4.0?edition=oss</a>Once Installed Grafana start server.<figure><img alt="" src="https://cdn-images-1.medium.com/max/854/1*qEOslxjOBtjSZFXTcaEr4w.png" /></figure>Login and Access Grafana: Access the Grafana web interface (usually http://&lt;VM_IP&gt;:3000)<figure><img alt="" src="https://cdn-images-1.medium.com/max/501/1*be8ga1a3AeMoZen9cQpuDw.png" /></figure>you can find login credentials in grafana.ini file.<figure><img alt="" src="https://cdn-images-1.medium.com/max/759/1*ZzUfDrm_D5nPFUXFk5cKHA.png" /></figure>Navigate to Connections -&gt; New Connection → Add Data source and Add below two datasource Plugins for Google Cloud Monitoring &amp; Google Cloud Logging.<figure><img alt="" src="https://cdn-images-1.medium.com/max/726/1*TWFCdXViyPIQUqUAolcJrw.png" /></figure>Set Up Authentication and Test the connection between Grafana and your GCP project to ensure data can be retrieved successfully.<figure><img alt="" src="https://cdn-images-1.medium.com/max/769/1*CJDQpGQMIoO5wl9hlR4XXw.png" /></figure>Importing Built-in Dashboards &amp; Building Custom DashboardsGrafana offers a wide range of pre-built dashboards for various GCP services. You can import these dashboards and customize them to fit your specific needs. Additionally, Grafana’s intuitive interface allows you to build custom dashboards from scratch, enabling you to visualize the metrics that matter most to you.<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-AxhY79Txip0bnUr8Ug7mA.png" /></figure>Sample Dashboards<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ATVPgDOZ9-9ONFhgxuYJVg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/812/1*NRdpAHU8EmO2mOC9FVWxog.png" /></figure>Logging<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rsjBfv2lshxLlKwyPWeoVQ.png" /></figure>Checking Errors<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*57LaIEKk_EkpgoR5OZnSzA.png" /></figure><blockquote>YouTube Video Demo</blockquote>For a more detailed visual walkthrough, check out this helpful YouTube video demonstration (link to your video here).<iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FZ7l4coV4KjE%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DZ7l4coV4KjE&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FZ7l4coV4KjE%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/cbd55ba6ea43a3c414c5111d8401c358/href">https://medium.com/media/cbd55ba6ea43a3c414c5111d8401c358/href</a></iframe><blockquote>Conclusion</blockquote>By setting up a self-hosted Grafana server on GCP, you gain a powerful tool to monitor and visualize your Google Cloud resources effectively. This approach offers greater control and customization while providing valuable insights into your cloud environment’s health and performance. Remember to adapt the security configuration and resource allocation based on your specific needs and environment.<blockquote>About Me</blockquote>As an experienced Fully certified (11x certified) Google Cloud Architect, Google Cloud champion Innovator, with over 7+ years of expertise in Google Cloud Networking,Data ,Devops, Security and ML, I am passionate about technology and innovation. Being a Champion Innovator and Google Cloud Architect, I am always exploring new ways to leverage cloud technologies to deliver innovative solutions that make a difference.If you have any queries or would like to get in touch, you can reach me at my email address <a href="mailto:vishal.bulbule@techtrapture.com">vishal.bulbule@techtrapture.com</a> or connect with me on LinkedIn at <a href="https://www.linkedin.com/in/vishal-bulbule/">https://www.linkedin.com/in/vishal-bulbule/</a>. For a more personal connection, you can also find me on Instagram at <a href="https://www.instagram.com/vishal_bulbule/?hl=en">https://www.instagram.com/vishal_bulbule/?hl=en</a>.Additionally, please check out my YouTube Channel at <a href="https://www.youtube.com/@techtrapture">https://www.youtube.com/@techtrapture</a> for tutorials and demos on Google Cloud.<img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5fdb8a6f27b1" width="1" height="1" alt=""><hr><a href="https://medium.com/google-cloud/monitor-google-cloud-with-self-hosted-grafana-5fdb8a6f27b1">Monitor Google Cloud with Self-Hosted Grafana</a> was originally published in <a href="https://medium.com/google-cloud">Google Cloud - Community</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.

Author

Link

Published date

Image url

Feed url

Guid

Hidden blurb

--- !ruby/object:Feedjira::Parser::RSSEntry
title: Monitor Google Cloud with Self-Hosted Grafana
url: https://medium.com/google-cloud/monitor-google-cloud-with-self-hosted-grafana-5fdb8a6f27b1?source=rss----e52cf94d98af---4
author: Vishal Bulbule
categories:
- gcp-security-operations
- grafana
- monitoring
- google-cloud-monitoring
published: 2024-05-15 00:49:47.000000000 Z
entry_id: !ruby/object:Feedjira::Parser::GloballyUniqueIdentifier
 is_perma_link: 'false'
 guid: https://medium.com/p/5fdb8a6f27b1
carlessian_info:
 news_filer_version: 2
 newspaper: Google Cloud - Medium
 macro_region: Blogs
rss_fields:
- title
- url
- author
- categories
- published
- entry_id
- content
content: '<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G_gcJC7qUPp_bUfFU90ECA.png"
 /></figure><blockquote>Introduction</blockquote>Managing and
 monitoring your Google Cloud Platform (GCP) resources is crucial for maintaining
 optimal performance and identifying potential issues. While GCP offers its own monitoring
 tools, integrating them with a powerful visualization platform like Grafana can
 take your monitoring game to the next level. This blog will guide you through setting
 up a self-hosted Grafana server on a GCP VM and leveraging it to monitor your cloud
 resources.<blockquote>What is Grafana?</blockquote>Grafana
 is an open-source platform that allows you to create interactive dashboards to visualize
 metrics, logs, and traces from various data sources. It excels at transforming complex
 data sets into easily digestible charts, graphs, and other visual elements, enabling
 you to gain deeper insights into your system’s health and performance.<figure><img
 alt="" src="https://cdn-images-1.medium.com/max/456/1*BQpgEIUMeRnsbSmX106A0Q.png"
 /></figure><blockquote>Grafana OSS vs Grafana Cloud</blockquote>Grafana
 comes in two flavors: Open Source (OSS) and Cloud. The OSS version offers complete
 control and customization but requires self-hosting and maintenance. Grafana Cloud
 provides a managed service with features like automatic scaling and enterprise-grade
 security, ideal for those seeking a hassle-free experience. You can find more
 details on Grafana Cloud vs OSS version here : <a href="https://grafana.com/oss-vs-cloud/">https://grafana.com/oss-vs-cloud/</a>In
 this article we are focusing Grafana OSS version i.e self hosted Grafana.<blockquote>Creating
 Your Own Grafana Server on a GCP VM</blockquote>Provision a GCP VM:
 Choose a suitable VM instance type based on your expected load.I created basic debian VM.Install
 Grafana: Follow the official Grafana installation guide for your chosen operating
 system (e.g., Ubuntu, Debian). You can find download instructions here : <a href="https://grafana.com/grafana/download/10.4.0?edition=oss">https://grafana.com/grafana/download/10.4.0?edition=oss</a>Once
 Installed Grafana start server.<figure><img alt="" src="https://cdn-images-1.medium.com/max/854/1*qEOslxjOBtjSZFXTcaEr4w.png"
 /></figure>Login and Access Grafana: Access the Grafana web interface (usually
 http://&lt;VM_IP&gt;:3000)<figure><img alt="" src="https://cdn-images-1.medium.com/max/501/1*be8ga1a3AeMoZen9cQpuDw.png"
 /></figure>you can find login credentials in grafana.ini file.<figure><img
 alt="" src="https://cdn-images-1.medium.com/max/759/1*ZzUfDrm_D5nPFUXFk5cKHA.png"
 /></figure>Navigate to Connections -&gt; New Connection → Add Data source and
 Add below two datasource Plugins for Google Cloud Monitoring &amp; Google Cloud Logging.<figure><img
 alt="" src="https://cdn-images-1.medium.com/max/726/1*TWFCdXViyPIQUqUAolcJrw.png"
 /></figure>Set Up Authentication and Test the connection between Grafana and
 your GCP project to ensure data can be retrieved successfully.<figure><img alt=""
 src="https://cdn-images-1.medium.com/max/769/1*CJDQpGQMIoO5wl9hlR4XXw.png" /></figure>Importing
 Built-in Dashboards &amp; Building Custom DashboardsGrafana offers a wide
 range of pre-built dashboards for various GCP services. You can import these dashboards
 and customize them to fit your specific needs. Additionally, Grafana’s intuitive
 interface allows you to build custom dashboards from scratch, enabling you to visualize
 the metrics that matter most to you.<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-AxhY79Txip0bnUr8Ug7mA.png"
 /></figure>Sample Dashboards<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ATVPgDOZ9-9ONFhgxuYJVg.png"
 /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/812/1*NRdpAHU8EmO2mOC9FVWxog.png"
 /></figure>Logging<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rsjBfv2lshxLlKwyPWeoVQ.png"
 /></figure>Checking Errors<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*57LaIEKk_EkpgoR5OZnSzA.png"
 /></figure><blockquote>YouTube Video Demo</blockquote>For a
 more detailed visual walkthrough, check out this helpful YouTube video demonstration
 (link to your video here).<iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FZ7l4coV4KjE%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DZ7l4coV4KjE&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FZ7l4coV4KjE%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube"
 width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/cbd55ba6ea43a3c414c5111d8401c358/href">https://medium.com/media/cbd55ba6ea43a3c414c5111d8401c358/href</a></iframe><blockquote>Conclusion</blockquote>By
 setting up a self-hosted Grafana server on GCP, you gain a powerful tool to monitor
 and visualize your Google Cloud resources effectively. This approach offers greater
 control and customization while providing valuable insights into your cloud environment’s
 health and performance. Remember to adapt the security configuration and resource
 allocation based on your specific needs and environment.<blockquote>About
 Me</blockquote>As an experienced Fully certified (11x certified)
 Google Cloud Architect, Google Cloud champion Innovator, with over 7+ years of expertise
 in Google Cloud Networking,Data ,Devops, Security and ML, I am passionate about
 technology and innovation. Being a Champion Innovator and Google Cloud Architect,
 I am always exploring new ways to leverage cloud technologies to deliver innovative
 solutions that make a difference.If you have any queries or would like to
 get in touch, you can reach me at my email address <a href="mailto:vishal.bulbule@techtrapture.com">vishal.bulbule@techtrapture.com</a>
 or connect with me on LinkedIn at <a href="https://www.linkedin.com/in/vishal-bulbule/">https://www.linkedin.com/in/vishal-bulbule/</a>.
 For a more personal connection, you can also find me on Instagram at <a href="https://www.instagram.com/vishal_bulbule/?hl=en">https://www.instagram.com/vishal_bulbule/?hl=en</a>.Additionally,
 please check out my YouTube Channel at <a href="https://www.youtube.com/@techtrapture">https://www.youtube.com/@techtrapture</a>
 for tutorials and demos on Google Cloud.<img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5fdb8a6f27b1"
 width="1" height="1" alt=""><hr><a href="https://medium.com/google-cloud/monitor-google-cloud-with-self-hosted-grafana-5fdb8a6f27b1">Monitor
 Google Cloud with Self-Hosted Grafana</a> was originally published in <a href="https://medium.com/google-cloud">Google
 Cloud - Community</a> on Medium, where people are continuing the conversation by
 highlighting and responding to this story.'

Language

Active

Ricc internal notes

Ricc source

Show this article Back to articles