Calculating and searching 500 billion digits of Pi

It's Pi day! Have you ever wondered whether you could take on the infinite irrationality of Pi and calculate its over one trillion digits? Well we wondered that exactly, and we found a way to calculate to 500 billion digits. Here's how:

Google Compute Engine supports up to eight 375GB Local SSDs per virtual machine instance! That gave us a total of 3TB of Local SSDs to use as swap space — exactly what we needed to calculate 500 billion digits, which is also exactly what we did.
500 billion digits of Pi on Google Compute Engine in 44.9 hours



Once the machine was setup, we were able to calculate 500 billion digits of Pi in about 44.9 hours. Assuming you already have a Google Cloud Platform account, and gcloud command line tool installed, here's how you can setup the instance:

  1. First, let’s set some variables to be used later. You’ll need to pick a zone to replace the ZONE variable.
    export PROJECT=”YOUR_GCP_PROJECT"
    export ZONE="us-central1-c"
    export CORES="32"
    export OUTPUT_DISK_SIZE="500GB"
    export OUTPUT_DISK="out-${CORES}"
    export INSTANCE="yc-${CORES}"
  2. Next, create a large disk to hold the final output (500 billion digits will take up 500GB!)
    $ gcloud compute disks create ${OUTPUT_DISK} 
     --project ${PROJECT} 
     --zone ${ZONE} 
     --size ${OUTPUT_DISK_SIZE} 
     --type pd-ssd 
  3. Then, create a Compute Engine instance with Local SSDs, the following command will attach 8 Local SSDs:
    $ gcloud compute instances create ${INSTANCE} 
     --project ${PROJECT} 
     --zone ${ZONE} 
     --machine-type n1-highmem-${CORES} 
     --maintenance-policy TERMINATE 
     --image-project gce-nvme 
     --image nvme-backports-debian-7-wheezy-v20151104 
     --local-ssd interface=NVME 
     --local-ssd interface=NVME 
     --local-ssd interface=NVME 
     --local-ssd interface=NVME 
     --local-ssd interface=NVME 
     --local-ssd interface=NVME 
     --local-ssd interface=NVME 
     --local-ssd interface=NVME 
     --disk name=${OUTPUT_DISK},device-name=${OUTPUT_DISK}
  4. Once the instance is started, you can SSH into it with:
    $ gcloud compute ssh ${INSTANCE} -zone ${ZONE}
  5. Once you're in the newly created instance, you'll need to first format and mount all of the Local SSDs and the large persistent disk:
    $ sudo su -
    $ for i in `seq 0 7`; do 
      mkdir /mnt/${i}; 
      /usr/share/google/safe_format_and_mount 
       /dev/disk/by-id/google-local-ssd-${i} /mnt/${i}; 
    done
    
    $ mkdir /mnt/out
    $ /usr/share/google/safe_format_and_mount 
       /dev/disk/by-id/google-${OUTPUT_DISK} 
       /mnt/out
  6. Finally, install the latest version of y-cruncher onto that instance. You may want to install screen or tmux as well.
    To calculate digits of Pi, here is the command line I used to start y-cruncher on the virtual machine:
    $ export DIGITS=”500000000000"
    $ ./y-cruncher custom pi -dec:${DIGITS} -hex:0 
      -o /mnt/out -mode:swap -swap:raid0 
      /mnt/0 /mnt/1 /mnt/2 /mnt/3 /mnt/4 /mnt/5 /mnt/6 /mnt/7

Searching Pi Digits (Preview!)

y-cruncher certainly made it easy to calculate digits of Pi. For Pi Day 2016, Francesc Campoy, Jen Tong, Sara Robinson and I built a reverse lookup index in Google Cloud Bigtable, so that we can search a sequence of digits, such as your phone number and more! Here is a sneak preview of 2-million writes per second of Pi Digits into Cloud Bigtable:
So that we can search a sequence of digits (up to 20 digits), and ask where in Pi does the first 9 digits of e appear? 6 occurrences! One in position 2,630,513,465!
Or, is there a sequence of nine 9’s “999999999”:
In position 4,329,769,635!

This was a big leap from last year, when we calculated 100 billion digits using y-cruncher running on last year's n1-highmem-32 of Compute Engine instance, which had 32-cores and 208GB of RAM and four Local SSDs.

Stayed tuned for more details on how we ingested billions of digits into Bigtable and how we built the Pi Search frontend.

- Posted by Ray Tsang, Developer Advocate and Greg Wilson, Head of Developer Advocacy for Google Cloud Platform