Use Homebrew with private repo
4 min read

Use Homebrew with private repo

Background

For an old programmer, you must have a lot personal private scripts and tools and want to install them on all hosts you manage. Writing Makefile or shell script to build and put these things properly is tedious and sometimes hard to versioning them. With Homebrew, life is getting easy.

Create a new private tap

What is a tap

tap is a Homebrew terminology which is usually a Github repo containing a lot of formulas. Formula describes a specific package than can be manipulated by the brew command.

For example, when using brew install neovim, the neovim is a formula name, and the neovim formula contains the detailed procedures for downloading, compiling and installing neovim to a predefined location. Formula is just a description of a package; it doesn’t contain any source code of that package. That means the package itself can be shipped from other developers, as long as you have the permission to access it.

Sum up:

  1. tap is a Github repo contains a series of formula files;
  2. formula files are ruby scripts saying how to download, compile and install a specific package;
  3. package can be either managed by you or third-parties, as long as you have right to access it.

Create a private tap

tap usually appears in a form of Github repo, and usually the repo is public. However, in this article, I want to use a private Github repo as tap. You must have enough permission to clone this private repo no matter with ssh or Github token.

The repo acting as a brew tap usually follows the naming convention homebrew-tapname . The repo name is prefixed with homebrew- , and brew can recognise it as tap d0u9/tapname . The d0u9 here is my Github account.

To create a new private brew tap, simply create a Github private repo and give it a name prefixed with homebrew-. That’s all.

As an example, I will use d0u9/homebrew-brew repo as an example.

After creating the repo, add it to your local brew:

# Because this is a private repo, I use ssh protocol here to clone it.
brew tap d0u9/brew [email protected]:d0u9/homebrew-brew.git

Create a formula

formula is description file written in Ruby script to tell how to download, compile and install a specific package.

Here I use another private Github repo d0u9/hello as an example to illustrate how to install this package via brew.

d0u9/hello contains nothing but a shell script prints “hello world” to stdout.

Create formula script

Implementing your new formula by inheriting the Formula class provided by Homebrew. I don’t want to talk about the verbose detail of how to inherit it, because the are enough articles talking about this. The Homebrew’s official Formula cookbook is a good reference for Formula writers.

Basically, you need to compose a Formula like this:

class Hello < Formula
  depends_on "ruby"
  desc "A hello world formula sample"
  homepage "https://en.wikipedia.org/wiki/%22Hello,_World!%22_program"
  url "[email protected]:d0u9/hello.git",
      using:  GitDownloadStrategy,
      branch: "master"
  version "0.0.0"
  license "BSD-2-Clause"

  uses_from_macos "ruby"

  def install
    raise NotImplementedError
  end

  test do
    system "false"
  end
end

Save this file as Formula/hello.rb .

The key points here are:

  1. url "[email protected]:d0u9/hello.git";
  2. using: GitDownloadStrategy.

GitDownloadStrategy is an officially shipped download strategy. More officially supported download strategies can be found here.

The GitDownloadStrategy knows about the url [email protected]:d0u9/hello.git, and it can download the source directly by using ssh protocol.

But, what can I do if the only way to access the repo is by using Github Token?

If you've ever searched for similar topics on Google, you may have encountered this. I can't provide much information about it because the shit mentions everything except the GitHubPrivateRepositoryReleaseDownloadStrategy. This download strategy isn't an official one, and the article doesn't mention anything about it. I'm genuinely curious if the author thoroughly reviewed the paper.

The straightforward method is using:

url: "https://<token>@github.com/d0u9/hello.git",

How can we avoid exposing our token to the Brew user and enable them to use their own GitHub Token for accessing the private repository?

Write your own download strategy

It appears that there is currently no official download strategy available for managing private packages as of the time of writing this article.

Goal

Our goal is that using https://github.com/d0u9/hello.git as the url, then try to use HOMEBREW_GITHUB_API_TOKEN environment variable as the Github Token to clone the repo. If this variable isn’t set, fallback to use ssh protocol.

Implementation

We have used GitDownloadStrategy in previous example. To implement our own download strategy, deriving GitDownloadStrategy is a simple way.

require 'download_strategy'

class GithubPrivateRepoDownloadStrategy < GitDownloadStrategy
  def initialize(url, name, version, **meta)
    # Make sure if the url is not a ssh protocol
    is_match = %r{^https?://github\.com/[^/]+/[^/]+\.git$}.match?(url)

    if is_match
      if ENV.key?("HOMEBREW_GITHUB_API_TOKEN")
        # Use token
        parts = url.split("://")
        url = "#{parts[0]}://#{ENV["HOMEBREW_GITHUB_API_TOKEN"]}@#{parts[1]}"
      else
        # Use git, We assume that we can clone in using SSH.
        url = url.sub(%r{^https://([^/]+)/}, 'git@\1:')
      end
    end

    # Init all
    super
  end
end

The code is self-explained, I don’t need to put more words on it. Save this as:

Library/Homebrew/github_private_repo_download_strategy.rs

The final Formula file

require_relative "../Library/Homebrew/github_private_repo_download_strategy"

class Hello < Formula
  desc "A hello world formula sample"
  homepage "https://en.wikipedia.org/wiki/%22Hello,_World!%22_program"
  url "https://github.com/d0u9/hello.git",
      using:  GithubPrivateRepoDownloadStrategy,
      branch: "master"
  version "0.0.0"
  license "BSD-2-Clause"

  depends_on "ruby"

  uses_from_macos "ruby"

  def install
    raise NotImplementedError
  end

  test do
    system "false"
  end
end

Check your formula

Before committing your change, using brew’s audit command to verify if it has some obvious errors:

brew audit --strict --new-formula --online

You may be asked to using username and password to login, and I don’t know how to overcome it. My solution is change url to ssh protocol to let the audit command check other parts, and then change the url back to the ssh protocol.

Install formula

brew install d0u9/brew/hello

Useful Commands

# Add a new tap
brew tap d0u9/brew [email protected]:d0u9/homebrew-brew.git
# Remove a tap
brew untap d0u9/brew

# Create a new formula for tar.gz file in tap d0u9/brew.
# --ruby here means using ruby helper functions
brew create --tap d0u9/brew --ruby https://github.com/xxxx/refs/tags/1.4.5.tar.gz

# List formulas in a tap
brew search d0u9/brew

# Install a formula
brew install d0u9/brew/hello
# Remove a formula
brew uninstall d0u9/brew/hello

References

  1. https://docs.brew.sh/Formula-Cookbook#url-download-strategies
  2. https://github.com/Homebrew/brew/blob/master/Library/Homebrew/download_strategy.rb
  3. https://docs.brew.sh/Formula-Cookbook#install-the-formula
  4. https://docs.brew.sh/Formula-Cookbook#just-moving-some-files