Learn how to safely remove exposed authentication tokens from your Git repository’s entire history using BFG, a powerful cleanup tool. Understand the security risks of committed secrets and master the step-by-step process to sanitize your code repository and protect your organization’s digital assets.

The Unexpected Security Alert

Picture this: You’re a developer, deep in the middle of a meaningful sprint, when suddenly a JIRA ticket lands in your inbox. The subject line sends a chill down your spine: “Critical Security Issue – Exposed NPM Token in Repository History.”

The automated security scan has uncovered something alarming – an authentication token accidentally committed to the repository years ago, buried in commits from 4, 7, and 8 years back. It might seem like ancient history, but in the world of cybersecurity, old secrets can be ticking time bombs. And your CISO can be very insistent on solving such issues (with merit…)

The Hidden Risks of Exposed Tokens

An exposed authentication token is more than just a minor oversight. It represents a significant security vulnerability that could potentially:

• Allow unauthorized access to private package registries
• Expose sensitive company infrastructure
• Provide a potential entry point for malicious actors
• Risk company intellectual property
• Potentially violate compliance requirements

The reputational damage of a security breach can far outweigh the momentary convenience of a hastily committed token.

The first thing you’d like to do in such a case is to “rotate” the secret key(s) involved. Just null them and create new ones. But the risk doesn’t end there, because as someone put it in a JavaScript group discussion – old obsolete keys might hold a clue to the key generator’s algorithm itself. I’m not a cyber security expert (not even close), so I’ll take the experts’ words on it. How do we get rid of the exposed key in the repo then?

Enter BFG: Your Git History Cleanup BFF

Our solution? BFG (Big Fast Git), a powerful tool designed to clean up Git repository histories quickly and efficiently. Here’s a step-by-step guide to token removal:

1. Tell everyone who’s working on the repository to hold their work for a few minutes until you push your changes.
2. Download BFG and cd into its folder.
3. Clone the repository with a mirror:

git clone --mirror https://github.com/your-name/your-repo.git

I suggest doing it twice to two different folders so you will have a backup of your “old” repository if something goes wrong.

4. Create a tokens.txt file with the replacement syntax:

old_token1==>PLACE_HOLDER1
old_token2==>PLACE_HOLDER2
...

This will tell BFG to replace any instance of the old tokens with their placeholders.

An example of this would be:

ab23234-89723c-aa7b7c==>${NPM_TOKEN}

5. Run BFG with the replacement file (make sure you are in the repository’s parent folder):

java -jar bfg-1.14.0.jar --replace-text tokens.txt your-repo.git --no-blob-protection

6. Clean up the repository:

cd your-repo.git
git reflog expire --expire=now --all && git gc --prune=now --aggressive

7. Check the token was replaced by cloning the local mirror and “grepping” it:

cd .. && git clone your-repo.git && cd your-repo && git grep -n "REPLACE _WITH_YOUR_TOKEN"

If this returns 0 results, it means it worked.

8. Force push the changes:

cd your-repo.gitgit push --mirror

9. Tell your team mates to fetch and apply the changes:

git fetch
git reset --hard origin/main 

That’s it. Your repository is now clean. From now on, I guess the same tool that alerted you about the tokens can raise the alarm during a PR process before the token “spreads” to the main branch. Make sure it does 😉

Pro Tips for Token Management

• Never commit tokens directly to repositories
• Use environment variables or secure secret management tools
• Implement pre-commit hooks to prevent accidental token commits
• Regularly audit your repository history
• Rotate tokens periodically, especially if you suspect exposure

Final Thoughts

In an era of increasing cybersecurity threats, proactive repository management isn’t just good practice—it’s a necessity. Tools like BFG provide developers with powerful mechanisms to clean up historical mistakes and maintain the integrity of their code repositories.

While we’ve focused on token replacement, BFG is a versatile tool capable of:

• Removing large files from repository history
• Cleaning sensitive information
• Reducing repository size
• Sanitizing commits before open-sourcing projects

If you adhere to the reasons you should use conventional commits, BFG can allow you to spice up your old commits with some meaningful version and changelog text.

Remember: In software development, what’s committed isn’t always forever, thanks to tools like BFG.