Editor's Note: Today we hear from our partner JFrog, which recently refactored its Artifactory SaaS offering onto Google Cloud Platform (GCP), making it possible for joint JFrog and GCP customers to co-locate their development and production environments. Read on for more details about how JFrog architected and optimized the service to run on GCP.
JFrog Artifactory SaaS is a universal artifact repository hosted in the cloud. Our customers use it as the one-stop-binary-shop for their Docker registry, and repositories for Maven, npm, PyPi, Nuget and more. It offers freedom of choice in several dimensions supporting all major package formats, CI servers and build tools. We recognized the need to add Google Cloud Platform (GCP) to offer more choice, and support organizations already using GCP so they could co-locate all their cloud services. We set up Artifactory SaaS hosted on GCP using Google Cloud Storage as its massively scalable object store.
JFrog Artifactory SaaS deployment architecture
While setting up an enterprise-grade cloud service can get complicated, GCP offers an extensive range of services to make it easier. The architecture we developed is constructed as four layers based on GCP services, mainly Google Compute Engine and Cloud Storage. The four layers are:- Network Load Balancers, balance requests coming from the outside world into the front-end web Nginx stacks
- Web servers based on stacked Nginx servers are responsible for internal application load balancing and proxying requests
- Artifactory application servers
- Data and Metadata Management using Google Cloud SQL and Cloud Storage. Cloud SQL manages the Artifactory internal database fundamental to the product’s checksum-based storage, and Cloud Storage is the application’s massively scalable object store, where all the binary artifacts are actually hosted.
To onboard new customers quickly and easily, we developed a scripted provisioning service. As soon as you register for a free trial, the JFrog Store triggers the provisioning mechanism to automatically set up and configure all layers in the service architecture, helping you get up and running virtually instantly. This structured and efficient onboarding mechanism made it easier for us to adapt our internal provisioning solution by simply swapping out API calls and replacing them with relevant GCP API calls such as those in the Compute Engine and Cloud Storage services.
This structured and efficient onboarding process made it easier for us to adopt GCP as a new additional underlying platform for the service.
The four tenets of an enterprise-grade service
In designing our setup, we wanted to ensure it would meet the requirements of any enterprise:Scalability
Every layer in the architecture can be quickly scaled by our provisioning mechanism to meet any load requirements. Whether adding Compute Engines to the network or web service layers, or adding Artifactory instances, any element can be scaled up on demand without incurring any downtime to the system, and storage scales automatically as needed.
High availability
Our years of experience with our Artifactory SaaS taught us that one of the most critical issues that enterprises are concerned about when considering a cloud service is availability. Since an artifact repository is a vital resource for a development organization, any downtime can quickly translate to dollars lost. Our architecture takes availability to the extreme with redundancy implemented at every level, resulting in a setup with no single point-of-failure. The whole system is redundantly deployed on multiple distinct zones, so even if there is a general failure in one zone, the system will failover to another zone and continue to provide service to customers. As demand increases for JFrog Artifactory SaaS on GCP, we have complete flexibility to quickly set up redundant installations in additional zones as needed and on-demand.
Disaster recovery capabilities
To support disaster recovery, we utilize the built-in capability for multi-region storage on Cloud Storage and replicate Artifactory’s Cloud SQL database that contains both application and customer data. This allows us to failover to a recovery region in case of an outage in the main active region with no noticeable impact to users.
Security
The system maintains a clear separation at every level for both customers on a dedicated Artifactory SaaS installation and those on multi-tenanted installations. Dedicated customer installations use separate virtual devices at each layer of the architecture for maximum security. For customers in a multi-tenant environment, our provisioning mechanism automatically creates clearly separated folders, Artifactory servers and web server configurations as well as a dedicated filestore in Cloud Storage buckets.
Lessons learned
While JFrog Artifactory SaaS was already a mature and robust service, we learned a few lessons while migrating the service to GCP.Tweak to peak
While a service may run on top of different infrastructures, each cloud provider has its nuances. To optimize JFrog Artifactory SaaS on GCP, we focused on tweaking resource allocation such as number of threads and more in each layer to get optimal performance.
Offload to buckets
Working with Cloud Storage buckets made it much easier for us to manage the service than when using traditional storage solutions such as NFS. Things like monitoring folder sizes and storage capacity was a non-issue since these functions are provided by Cloud Storage buckets. On the whole, our service got “lighter.”
Setting up JFrog Artifactory SaaS on top of GCP was a great decision for us. Our past experience with Artifactory SaaS helped us in migrating and modifying our cloud service to a new platform while maintaining the same high quality of service. As a leader in binary artifact management, we take "universality" as a guiding principle, and believe that hosting our service on GCP is a great way to serve our customers. We'll continue to grow with Google Cloud as more services are added to enhance scalability, availability and reliability.
Open source developer? Get a free ride
JFrog’s tools are made by developers for developers. We are part of the OSS community and strive to provide it with the best vehicle to ride in. Together with the Google Cloud team, JFrog is happy to sponsor repositories free of charge (including Artifactory SaaS on top of a JFrog sponsored GCP infrastructure) for open source projects. Browse to the registration form and feel free to submit your request.
Read more about JFrog Artifactory on GCP. And may the frog be with you!
