It’s been a few years since I’ve last commented on my deployed Kubernetes-based workbench. Over that time I’ve learned a lot and gotten more experience with it and other solutions - it’s time to revisit the setup and decide whether to upgrade or replace.

View comments.

Before we can begin in earnest there are just a few requirements we need to put together, but many of you will already have these in place - an AWS account, a github account, and a domain name. If you don’t we’ll have you sorted out in just a …

View comments.

Now we’ll start getting to the fun part - actually deploying our infrastructure and launching a basic jupyterhub deployment. While I’ve simplified many steps, I’ve left many other intentionally exposed - to offer opportunities for customization and to expose how things work. What follows is a condensed summary of …

View comments.

We’ve sketched out a rough architecture at this point, time to get an estimate of expected cost, and think through how well we can expect it to meet our original design goals. Besides overall user experience, I find it’s important to carefully think through the limitations of any …

View comments.

Now that we have some of our design goals in place for the new environment we can start evaluating our options and creating the solution architecture. It will be critical to balance cost, performance, ease of administration, and ease of use in order to make this a truly viable platform …

View comments.

One of the most daunting tasks of getting going on a personal data science project can be just getting started with the environment. What I generally want is instant access to the latest technology stack (hopefully at low cost), allowing me to get to work quickly while also exploring the …

View comments.

One of the hardest parts of learning Deep Learning is just figuring out how to start! A particularly annoying part of the process is getting access to a GPU environment - GPUs aren’t absolutely necessary for learning DL, but they are an essential part of making problems feasible to solve …

View comments.

Installing a container orchestration layer will allow us to quickly deploy anything in the huge ecosystem of available docker services. This can include database servers like Postgres or MariaDB, message brokers like RabbitMq, medical informatics servers like HAPI FHIR, or our own services. There are several options, including Kubernetes, Docker …

View comments.

We have several options for a data science front end, but my favorite is Jupyter Hub. Jupyter is the rebranded version of IPython, now generalized beyond Python to include R, Julia, Scala, and many other languages. Jupyter notebook provides a web-based interface that will allow us to create analytics experiments …

View comments.

Now that we have a secure private network and a bastion node setup, we can start configuring internal servers. Our goal is to configure as few servers as possible - not only is this easier (hooray for lazy sys admins!), it’s also more repeatable and more secure as it’s …

View comments.