Bitbucket Secret Scanning (Step-by-Step)

In a rapid, automated DevOps environment, security teams struggle to ensure all aspects of code deployment are safe from exploits. They might scan software for vulnerabilities, but what's often forgotten is to scan the source code itself for credentials (such as API keys, private keys, database credentials, etc.) before committing it to Git repositories.

API secrets, keys, credentials, and access tokens should never be hardcoded in a project's source code. Still, it's common for developers to include them in their code during development out of convenience. In some scenarios, developers might know that secrets should not be stored in code, but they might accidentally push sensitive data to git repositories during their commits. Once secrets are sent to a Bitbucket server, they will remain in the commit history and should be assumed to be compromised. The only effective remediation is to rotate the keys, update passwords, etc.

Soteri's Security for Bitbucket will help developers and operations people avoid adding secrets to git repositories. It's an automated tool that can be integrated with your DevOps and DevSecOps teams to help them find sensitive data in code and remediate it immediately. Several compliance standards, government regulations, and "shift left" security best practices suggest using automation tools for repeatable steps in the software development lifecycle (SDLC).

Proactive security and automation significantly reduce the risks of a data breach. You could find a scanner that reactively finds potential risks, but Soteri integrates directly with Bitbucket so that you can automate scans and block code where secrets are stored during pushes and commits to a repository. It's most beneficial to DevOps teams with numerous projects. Just one oversight and corporate secrets could be exposed to anyone with access to the Bitbucket server.

Here are the steps to scan for Git secrets using the Soteri Bitbucket vulnerability scanner.

Before you start, you need to be using a Bitbucket data center or Bitbucket server. The steps and images you see are from a Bitbucket server instance.

Steps to Enable Secret Scanning On Your Bitbucket Repository

1. Install Bitbucket App to Scan Repository

Soteri's Security for Bitbucket is available on the Atlassian Marketplace, and you need to install it to begin your secret scanning. You can download and try the scanner for free or pay for licensing upfront. Click the "Installation" tab for instructions on how to install Soteri's Security for Bitbucket. 

The next steps assume you've installed the product successfully, but you can contact support if you need help or have any questions.

2. Review the Secret Scanning Settings

After installing Soteri's app, you can get to your dashboard anytime from Bitbucket using the lock icon in the top-right corner of the dashboard. All the steps below start from the main app dashboard and are accessible by clicking the lock icon.

Engineers at Soteri built the product to support security for development projects out-of-the-box, so most organizations will find that default settings are sufficient for effectively scanning their Git repositories for sensitive information. If developers and security teams need more flexibility with controls and configurations, Soteri supports their unique use cases by offering custom rules integration.

To view the app's rules, click the gear icon on the main dashboard.

Before you decide to change the app's configurations, check out the default rules that are available out of the box. As you can see, Soteri covers many secrets that might be found within enterprise codebases, such as API keys, cloud credentials, SSH keys, and financial information. Numerous built-in rules are enabled during installation, but you can view and disable them too.

Custom security scanning is available to add specific scanning rules to your Security for Bitbucket instance for unique use cases. Let's say you have a codebase that uses your internal API or a project that authenticates with internal network credentials; you can create custom scanner rules using regex patterns to scan the codebase for any instances of sensitive information that aren't covered by built-in rules.

In this example, Soteri is customized to find any Bitcoin addresses stored in code.

3. Go to the New Soteri Global Security Dashboard

With your custom rules configured, you can now scan repositories. You can scan code from three places:

• From the repository page
• From the project page
• From the Bitbucket global dashboard

The app's main dashboard lets you scan a single repository or all repositories on the Bitbucket server. For most security teams, a scan of all repositories is necessary, but developers will only be interested in scanning their own projects. Both options are available for their respective use cases.

4. Click "Scan Whole Instance" to Scan All Projects

Although you can scan individual projects and repositories, we recommend using the scan on the main global dashboard. In Soteri's Security for Bitbucket, you'll see the Scan Whole Instance button to start the scan across all projects and repositories.

After you click to scan your Bitbucket instance, the scan is queued. The amount of time to scan the entire Bitbucket server depends on the size of repositories and the number of projects you store. You can view active queued scans by clicking Active scans in queue and viewing the status of the most recent one that you just initiated.

Scanning a single project at a time is more common and recommended. Scanning a single project preserves resources and doesn't require a full scan of all repositories before completing. Most developers and security teams will prefer this scanning method to get quick insights into the health of a particular project. Developers can gain insight into their projects without waiting for other project scans to complete.

5. View or Export Vulnerabilities

After a vulnerability scan, you need to review reports. A basic overview can be seen in the main app's dashboard, where the Vulnerabilities Found column displays the number of issues found during the scan. The green line indicates no detected secrets, but a red line indicates that Soteri found at least one instance of sensitive information stored in source code. To get a report, click the Export option from the Actions menu.

The export feature stores every finding in a Comma Delimited Value (CSV) file you can download. The CSV file lists:

• Project name
• Repository
• Branch
• Commit
• File
• Line number
• Text snippet
• Type of sensitive information
• Allowlisted
• Reviewed

Sometimes, the secret detected is a false positive and does not present a security concern. Soteri will flag it in every scan which presents a false positive during a review. You can mark any value in your code as reviewed so that Soteri no longer flags it as a security risk. False positives will therefore no longer count as detected secrets.

From the main scan dashboard, click a project that Soteri found to store sensitive information. The project will display a list of repositories where secrets were stored. Click the repository you'd like to review, and Soteri displays the number of secrets found next to each branch name.

Click the number in the Vulnerabilities Found column for the branch you'd like to review, and a new tab opens that displays a list of each secret detected and its value. Browse the list of secrets found, but you can mark any one of them as a false positive by clicking the Mark Reviewed button.

6. After Retroactive Scan, Go Preventative

Retroactively scanning repositories is a great way to catch up to any security issues in your existing systems and git history, but any issues found during secret scanning mean that secrets were already exposed. As part of a "shift left" security initiative, DevOps teams should take a more proactive approach to secure their projects.

Every once in a while, retroactive scans can be done during an audit and security review, but Soteri can also be used to reject code during the commit process to stop secrets from ever entering the system. Hooks can be used to prevent a commit containing sensitive information from being pushed to Bitbucket. Alerts in hook configurations can be used to:

• Completely block new commits from being pushed to the Bitbucket repository, including pull requests. This will ensure dangerous code changes do not leave the developer's system.
• Only warn the developer pushing to the Bitbucket server but still allow the commit to happen.

Hooks can be configured in the Bitbucket security settings from the main Soteri scan dashboard. Soteri allows you to set up hooks for all repositories or enable them on a per-project basis, allowing custom repository settings.

Monitor the Audit Log

To help administrators keep track of various events, Security for Bitbucket places certain entries in Bitbucket's Audit Log.

The following events are currently recorded in the Audit Log:

• Changes in the Global Hook status or mode
• Built-in rule toggles
• Custom rule creation, deletion, changes, or toggles
• Findings being marked or unmarked as reviewed (false positives)
• Per-repository settings toggles
• Individual or group access changes to the Global Settings
• Changes to soteri-security.yml on the default branch of a repository
• Skipping the Soteri security hook by way of the specialized commit message skip-soteri-security-check

Here are several example records from the audit log, including details like IP address, Node ID, method, and description.

Run Your Scans Free for 30 Days

If you don't have a secret scanning solution to find vulnerabilities in your software, your organization is at risk of a data leak and breach. Storing secrets in code is a security concern that starts at the software development lifecycle. With Soteri's Security for Bitbucket, you can proactively scan your code, block developers from pushing secrets to their Bitbucket repositories, and protect sensitive data from being exposed.

To get started, check out the Soteri scanner in the Atlassian Marketplace and try it for free. Let us protect your source code and software development lifecycle from potential security risks.