Secret Scanning Tools: Best Practices Across the Software Development Lifecycle

Exposed API keys, database credentials, and access tokens are one of the most common causes of security breaches. A single leaked secret can lead to data breaches, account takeovers, service disruptions, brand damage, and significant financial penalties. Despite this risk, secrets routinely find their way into code repositories, build logs, and production environments.

Modern DevSecOps practices emphasize a shift-left approach: catching secrets as early as possible to minimize risk and remediation effort. However, implementing effective secret scanning requires more than a single point solution, and the tools for code scanning included with major VCS platforms are not thorough enough on their own. Security teams need a comprehensive, multi-layered defense strategy that provides continuous protection throughout the software development lifecycle (SDLC).

This article outlines considerations for where to integrate secret scanning tools to your development processes, with specific recommendations for each stage. We'll examine platform-specific capabilities across GitHub, GitLab, and Bitbucket, provide guidance on managing false positives, and share best practices for security teams to implement effective controls without hindering developer productivity.

Where to Implement Secret Scanning Tools in the SDLC

Within the SDLC, there are five main integration points where tools for secret scanning can be incorporated:

1. Local Development (Pre-Commit Hooks and IDE Plugins): Scanning for secrets before code is committed on a developer's machine (e.g., via Git pre-commit hooks) or as code is written (e.g., IDE plugins).
2. Pre-Push Checks: Scanning during the push process (client-side or server-side) to intercept secrets before they are added to remote VCS history. This often involves pre-push hooks or server-side push protection features that block the push if secrets are detected.
3. Pull/Merge Request Scanning: Analyzing code as part of code review (pull requests on GitHub/Bitbucket or merge requests on GitLab) to detect secrets in proposed changes. Alerts at this stage can stop a secret from being merged into the main branch.
4. CI Pipeline Scanning: Integrating secret detection into Continuous Integration (CI) pipelines. This runs scanners on committed code (and sometimes build artifacts) during automated builds/tests, catching secrets post-commit but before deploy.
5. Post-Deployment & Runtime Monitoring: Continuously monitoring deployed applications, logs, and runtime environments for secret exposure (especially dynamic secrets that only exist at runtime). This can catch secrets that leak in logs, environment variables, or other outputs after deployment.

The earlier in the SDLC a secret is caught, the lower the risk of spreading and the easier the remediation. Earlier checks (like pre-commit) require more developer-side enablement, while later checks (like runtime monitoring) serve as a safety net for secrets that slip through.

Let's go into each stage in more detail.

Key Integration Points for Secret Scanning

Local Development – Client-Side Secret Scanning

At the earliest stage, developers can leverage local tools like IDE plugins or pre-commit hooks to scan their code for secrets before a commit is finalized. This client-side approach prevents sensitive data from entering the version control history by providing immediate feedback to the developer.

Implementation

Secret scanning tools like Yelp's detect-secrets, AWS's git-secrets or gitleaks can be configured as Git pre-commit hooks. These tools search the code diff for patterns (like API key formats or high-entropy strings) and reject the commit if a potential secret is found. Many teams use the open-source pre-commit framework to centrally manage hook installation across developers.

Some secret scanning tools offer IDE plugins that provide real-time scanning directly within the editor. While pre-commit hooks run specifically during the git commit command, IDE plugins scan code as it's written or saved, offering earlier feedback.

Developer Experience

IDE plugins offer instantaneous feedback directly within the editor, a strong advantage for fixing issues immediately. Pre-commit scanning offers quick feedback upon attempting a commit. Both approaches require individual developer setup or organizational enforcement (e.g., via repository settings or documentation). Properly configured, the impact on a developer's workflow can be minimal, but careful tuning is needed to minimize false positives, as too many interruptions can lead developers to disable the tool or bypass the hook.

Native Platform Support

None of the major VCS platforms offer client-side hooks or IDE plugins out-of-the-box; you will need to integrate third-party tools or custom configurations. GitHub, GitLab, and Bitbucket all permit local Git hooks, but configuration is up to individual teams or developers. Similarly, IDE plugin usage is often managed by the individual developer.

Local scanning via hooks or IDE plugins is a client-side strategy providing the earliest intercept point, contingent on developer buy-in and maintenance.

Pre-Push Secret Scanning

Pre-push secret scanning occurs when developers attempt to push commits to the central repository. This can be implemented client-side (local pre-push hook) or server-side (pre-receive hook rejecting the push).

Note that if a secret was committed locally, even if the push is blocked, it still exists on the developer's machine, so the developer will need to purge the secret from their local commit history. Security and DevSecOps teams should provide clear documentation on these git operations since they can be complex for developers unfamiliar with history rewriting.

Client-Side Pre-Push Hooks

Even without server support, teams can set up local pre-push hooks (similar to pre-commit hooks) that run a secret scan on the outgoing commits. This is a less common pattern than pre-commit, but some organizations use it as an extra safety net. A pre-push hook can scan all commits in the push and abort if a secret is found. However, this still relies on local configuration (and can be bypassed).

Generally, the server-side approach is preferred for consistency, with GitHub and GitLab's native push protection being prime examples of server-side enforcement.

Server-Side Push Protection

Two out of the three most common VCS platforms offer server-side push protection out-of-the-box:

GitHub Advanced Security

GitHub's push protection (available with GitHub Advanced Security offers real-time scanning of pushed commits before they are accepted into the repository. When enabled:

1. It checks for over 200 token types and secret patterns
2. Push operations are blocked automatically if secrets are detected
3. Developers receive immediate feedback through the Git CLI or web interface
4. Admins can configure which secret patterns trigger blocking vs. alerts
5. Repository admins can bypass blocks when necessary (with justification)
6. Custom secret patterns can be defined for organization-specific tokens

For private repositories, this requires the Advanced Security license. For public repositories on GitHub.com, this capability is available for free.

GitLab Ultimate

GitLab's push rules (available in Premium and Ultimate tiers) provide similar functionality:

1. Administrators can enable secret detection at the instance, group, or project level
2. Detects over 50 common secret patterns including API keys, tokens, and passwords
3. Provides immediate feedback to developers through the Git CLI
4. Allows custom regex patterns to catch organization-specific secrets
5. Can be configured to block the push or simply alert for later remediation
6. Detailed logging of blocked pushes for security team review

GitLab's implementation integrates with their broader security dashboard, allowing security teams to track trends in secret detection across projects.

Bitbucket

Bitbucket lacks native server-side push protection for secrets, so teams typically implement custom hooks or third-party integrations:

1. Custom pre-receive hooks can be configured to reject pushes with secrets
2. Third-party apps like our Security for Bitbucket provide dedicated push protection
3. Some organizations use repository and branch restrictions combined with PR enforcement

Read more about secret scanning in Bitbucket and the limitations of their native solution in our blog post.

Developer Experience Considerations

Server-side push protection offers several advantages over client-side approaches:

1. Consistent enforcement: Rules are applied uniformly across all developers
2. Immediate feedback: Developers learn about issues at push time, before code is shared
3. No local configuration: Developers don't need to install or maintain hooks
4. Bypass mechanisms: For urgent situations, most systems allow authorized overrides

Pull/Merge Request Scanning

Pull requests (PRs) and merge requests (MRs) serve as a checkpoint before code merges into important branches. Scanning for secrets here ensures reviewers and developers address exposures before merging.

The primary disadvantage is that secrets have already been committed to the branch by this point, requiring additional remediation steps like history rewriting if sensitive information is found. Secrets found at this stage should be considered already compromised, and revoked or rotated as appropriate.

Implementation

Secret scanning at the PR/MR stage builds on repository scanning but narrows the focus to just the changes being proposed. This stage provides a final review opportunity before code reaches the main branch, and integrates directly with the code review process.

Major platforms implement this in different ways:

GitHub Advanced Security

GitHub's secret scanning for PRs automatically analyzes the code changes and adds annotations directly in the PR interface if secrets are detected. These appear as comments in the code diff and in the PR conversation. Reviewers can see these alerts while reviewing the code, making them part of the approval workflow. Alerts can be resolved or marked as false positives within the PR interface.

GitLab Ultimate

GitLab's secret detection runs as part of the MR pipeline, scanning the diff specifically for introduced secrets. Results appear in the MR widget, showing the severity, location, and type of secret detected. GitLab provides interactive feedback allowing developers to mark false positives or resolve findings. For GitLab Ultimate users, MR approval rules can be configured to require all secret findings to be resolved before merging.

Bitbucket

Bitbucket's native capabilities are more limited. While Code Insights can be used to display scanning results from external tools, there's no built-in secret scanning specific to PRs. Teams typically integrate third-party tools that run scans and publish results as Code Insights, or use Bitbucket Pipelines to run scans that post comments on the PR.

Continuous Integration (CI) Pipeline Scanning

Within Continuous Integration pipelines, secret scanning acts as an automated audit, serving as a safety net to catch secrets missed by earlier scans, as well as those introduced later in the development lifecycle. Secrets can be introduced after pre-commit, pre-push, or pre-PR steps through build-time generated credentials, automated scripts, or pulled in via dependencies and third-party packages that weren't part of earlier code reviews. It can also scan build artifacts.

Integration in CI

Secret scanning tools are typically run as a CI job, usually as part of a static analysis stage. All major platforms support this capability:

1. GitHub Actions: Teams can use community actions like Gitleaks Action or run Docker containers with secret scanning tools in their workflow.
2. GitLab CI: GitLab provides a built-in CI template for Secret Detection that adds a scanning job to your pipeline. In Ultimate tier, findings appear in the pipeline and security dashboards.
3. Bitbucket Pipelines: Add pipeline steps to run secret scanning tools, with output in SARIF or JSON format for reporting.

Similar approaches work with other CI systems like Jenkins, CircleCI, and Azure DevOps.

Pipeline Scanning vs. Pre-Push

Pipeline scans run after code is already in the repository. They provide an important safety net, potentially catching:

1. Secrets missed by push protections
2. Tokens not matching known patterns
3. Secrets introduced before push protection was enabled
4. Secrets in dependencies or generated during the build

The key trade-off: by the time CI flags a secret, it's already in the git history. This means the secret must be rotated and possibly purged from git history.

Post-Deployment: Runtime Monitoring for Secrets

Despite rigorous pre-merge scanning, secrets can surface during runtime (e.g., in logs) or be introduced dynamically. Post-deployment monitoring addresses these risks, especially for dynamic secrets not present in the codebase.

Runtime Secret Exposure

Common scenarios for runtime leaks include:

1. Logging of secrets (e.g., printing an AWS access key in a debug log)
2. Storing secrets improperly in files
3. Leaving credentials in publicly accessible endpoints

Sophisticated attackers could scan not just code repos but also container images, config files, and even CI/CD logs for exposed secrets.

Some approaches:

Log Monitoring for Secrets

Implement a DLP (Data Loss Prevention) style scan on application logs and CI/CD logs. For example, aggregate logs in a system like Splunk or ELK and set up detectors for secret patterns (AWS keys, tokens, etc.). OWASP recommends assembling logs from CI/CD tooling and having rules to detect secret usage or exfiltration attempts.

If a secret gets printed (even encoded or obfuscated), advanced regex patterns or high-entropy string detection might catch it. Some CI platforms automatically mask known secrets in logs, but this masking isn't foolproof—it can fail when secrets are transformed, encoded, or manipulated, resulting in false negatives. Because automatic masking can miss these cases, independent monitoring remains valuable.circumvented by transformations. So independent monitoring is valuable.

Production Monitoring & Scanning

Deployed infrastructure and artifacts can be scanned for secrets in a few ways:

1. Scanning Docker container images for hardcoded passwords or keys before deployment
2. Scanning VM images or server filesystems for leftover keys
3. Checking that environment variables in production don't contain unexpected secrets

Tools like Trivy (container scanner) have checks for secrets in images. Cloud providers offer services to detect leaked secrets – e.g., AWS config rules to flag hardcoded access keys or AWS Macie to scan data stores for sensitive data.

Ensure any config files (Kubernetes manifests, Terraform scripts) deployed to production have no hardcoded secrets – these should be caught earlier, but a double-check in deployment reviews can be wise.

Dynamic Analysis in Test Environments

Use dynamic application security testing (DAST) tools or custom scripts in a staging environment to see if the application is inadvertently exposing secrets. For instance, Aikido Security suggests using tools like OWASP ZAP or Burp Suite to observe runtime behavior and responses to detect exposure of secrets that static analysis might miss. If an API endpoint returns some secret on error, a dynamic scan could catch that.

Secret Usage Monitoring

Monitor the usage of credentials. For example, if a particular API key should only be used by a CI pipeline, but it's suddenly used from a different IP or context, that could indicate a leaked secret. Some secrets management tools support usage logs or can send alerts on anomalies.

Automatic Revocation

In cases where a secret is leaked, having automation to revoke it quickly reduces damage. GitLab can automatically revoke certain kinds of secrets when detected (for example, an AWS key via integration). Evaluate integrating your scanning alerts with scripts or provider APIs to auto-disable keys (with care to avoid false positives causing disruption).

Platform Comparison: Secret Scanning Capabilities Across Major VCS Providers

To help teams evaluate their options based on their current VCS platform, the following table compares secret scanning features across GitHub, GitLab, and Bitbucket:

GitHub and GitLab provide more comprehensive native secret scanning capabilities, particularly in their premium tiers. Bitbucket users will need to rely more heavily on third-party tooling or custom integrations to achieve similar coverage.

Recommendations for Security Leads and DevSecOps Engineers

Here are our recommendations for implementing secret scanning tools:

1. Prioritize a Multi-Layered Program: Architect a program leveraging secret scanning tactics at multiple stages within the SDLC, and at least one secret scanning tool at the pre-commit or pre-push stage so secrets are caught before they enter your codebase.
2. Maximize Platform Capabilities: Use built-in VCS features as available (like GitHub/GitLab push protection, GitLab MR widgets) as they are well integrated and low maintenance, supplementing with open source & third party tools as needed.
3. Tailor Scanning Rigor to Risk: Apply stricter enforcement (blocking actions, mandatory hooks) to high-risk repositories (IaC, production deployment paths). Use alerting modes for lower-risk projects, aligning friction with potential impact. However, enable basic detection broadly.
4. Address Dynamic Secret Risks: Plan for secrets outside the codebase. Collaborate with Ops/Cloud teams on log scanning, IaC analysis, and runtime configuration checks. Promote the use of secrets managers.
5. Institute Continuous Improvement: Treat scanning as an evolving program. Regularly review metrics (detection rates per stage), update patterns, revisit exclusions, solicit feedback, and assess if the program is effectively reducing incidents over time. Adapt policies based on evolving threats and internal needs.

Following these recommendations will allow security teams to leverage tools for secret scanning that reduce risk without slowing development velocity.

Soteri's Bitbucket Secret Scanning Solution

For Bitbucket Data Center teams looking to achieve comprehensive secret scanning coverage, Soteri's Security for Bitbucket app fills critical gaps left by native platform scanning. Our solution safeguards your repositories throughout the SDLC, proactively preventing secrets from entering your codebase with server-side push protection as well as on-demand scanning of your existing repos.

The app also enhances visibility and simplifies workflows with intuitive dashboards, easy data exports, and over 40 built-in detection patterns—streamlining your security team's workload.

Try Security for Bitbucket free for 30 days on the Atlassian Marketplace.

‍