Automating Shadow API and BOLA Detection with Custom Nuclei Templates

Automating the detection of Shadow APIs and Broken Object Level Authorization (BOLA) vulnerabilities is critical for maintaining robust API security. These often-overlooked flaws can expose sensitive data and critical functionalities, making their programmatic discovery a high-priority task for any offensive security professional. Custom Nuclei templates offer a flexible and powerful solution to identify these issues at scale, moving beyond generic checks to target application-specific logic and undocumented endpoints.

Shadow API Detection: The Phantom Menace

Shadow APIs are undocumented, forgotten, or intentionally hidden endpoints that operate outside official visibility and control. They emerge frequently in rapid development cycles, often exposing internal functionalities or old versions that bypass security scrutiny. Detecting them manually is tedious and rarely comprehensive.

Our objective is to uncover these lurking endpoints. This involves more than just scanning for known CVEs; it requires intelligence gathering and creative fuzzing based on common API naming conventions, observed patterns in existing documentation (if any), and directory enumeration. Tools like ProjectDiscovery's httpx and katana can identify live web servers and crawl for URLs, providing a solid foundation for Nuclei's targeted probes.

Crafting Nuclei Templates for Shadow API Discovery

A Nuclei template for Shadow API detection focuses on probing common API paths and attempting to access endpoints that might not be explicitly linked but follow predictable naming schemes. The template uses raw HTTP requests to send specific probes and then employs matchers to identify interesting responses, such as a 200 OK status code with JSON content, a 302 Redirect indicating an active but unlinked resource, or even 401 Unauthorized/403 Forbidden, which still confirm the endpoint's existence.

id: shadow-api-common-paths
info:
  name: Shadow API - Common Path Probes
  author: yourname
  severity: info
  description: Probes for commonly exposed or undocumented API paths (e.g., admin panels, unlinked versions).
  tags: shadow-api, discovery, api, low-hanging-fruit

http:
  - raw:
      # Common API root paths and versions
      - |
        GET /api/v1/admin/status HTTP/1.1
        Host: {{Hostname}}
        User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36
      - |
        GET /api/v2/health HTTP/1.1
        Host: {{Hostname}}
        User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36
      - |
        GET /rest/users/all HTTP/1.1
        Host: {{Hostname}}
        User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36
      # Potential admin dashboards or sensitive endpoints
      - |
        GET /admin/dashboard HTTP/1.1
        Host: {{Hostname}}
        User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36
      - |
        GET /management/api/users HTTP/1.1
        Host: {{Hostname}}
        User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/533.36
    
    matchers-condition: or
    matchers:
      - type: status
        status:
          - 200
          - 302
          - 401
          - 403
      - type: word
        words:
          - "json"
          - "data"
          - "id"
          - "API Version"
          - "admin panel"
        condition: or
        part: body

This template is basic. For advanced Shadow API discovery, combine this with subdomain enumeration and URL crawling to generate a comprehensive list of potential API endpoints. Fuzzing specific path, query, header, cookie, or body parts within Nuclei templates can further enhance discovery.

BOLA Detection: Authorization Breakdown

Broken Object Level Authorization (BOLA), often referred to as IDOR (Insecure Direct Object Reference), occurs when an API endpoint accepts an object ID as a parameter (e.g., /api/v1/users/123) but fails to properly verify if the authenticated user is authorized to access the requested object. This allows attackers to manipulate IDs to access data belonging to other users or retrieve objects they shouldn't. BOLA is consistently ranked as a critical API vulnerability due to its ease of exploitation and potential for significant data breaches.

Designing Nuclei Templates for BOLA

Automating BOLA detection requires a multi-step approach within Nuclei. First, you need to identify an endpoint that handles object IDs. Then, you typically make an initial authenticated request to extract a valid, but unauthorized, object ID (e.g., another user's ID). Finally, a subsequent request attempts to access this extracted ID using the attacker's (or a different, less privileged user's) session. The key is to analyze the response for indicators that unauthorized access was granted.

id: bola-chained-id-manipulation
info:
  name: Broken Object Level Authorization (BOLA) - Chained ID Manipulation
  author: yourname
  severity: high
  description: Attempts to detect BOLA by extracting an object ID and then manipulating it to access unauthorized resources.
  tags: bola, authorization, idor, api, critical

http:
  - raw:
      # Request 1: Authenticated request to get a list of items to extract a target ID.
      # Replace YOUR_VALID_SESSION_ID_HERE with a real, authenticated session cookie.
      - |
        GET /api/v1/users HTTP/1.1
        Host: {{Hostname}}
        User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36
        Cookie: session_id=YOUR_VALID_SESSION_ID_HERE
    
    extractors:
      - type: regex
        name: victim_user_id
        regex:
          - '"id":\s*"(?P<id>\d+)"' # Extracts the ID of the first user found
        group: 1
        internal: true # Marks the extracted ID for internal use in subsequent requests
        # Filter out your own user ID if known to avoid self-IDOR
        # dsl: |
        #   len(victim_user_id) > 0 && victim_user_id != "YOUR_OWN_USER_ID_HERE"

  - raw:
      # Request 2: Attempt to access the extracted victim_user_id with the current user's session.
      # Ensure 'YOUR_VALID_SESSION_ID_HERE' and 'YOUR_OWN_USER_ID_HERE' are correctly set.
      - |
        GET /api/v1/users/{{victim_user_id}} HTTP/1.1
        Host: {{Hostname}}
        User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36
        Cookie: session_id=YOUR_VALID_SESSION_ID_HERE
    
    matchers-condition: and
    matchers:
      - type: status
        status:
          - 200 # Expect a 200 OK if access is granted inappropriately
      - type: word
        words:
          - '"username":"victim_username"' # Look for an indicator of the victim's data (e.g., their username)
        part: body
        case-insensitive: true
      - type: word
        words:
          - '"username":"YOUR_OWN_USERNAME_HERE"' # Ensure it's not showing your own username
        part: body
        negative: true # Negative match to confirm it's not your own data

This template demonstrates a typical chained BOLA detection. The first request retrieves a list of users (e.g., from an endpoint like /api/v1/users) and extracts a victim_user_id using a regex extractor. The internal: true flag ensures this variable is passed to subsequent requests. The second request then attempts to access /api/v1/users/{{victim_user_id}} using the attacker's session. Positive matches include a 200 OK status code and the presence of the victim's data, while a negative match on the attacker's own username helps confirm it's another user's data.

Remember to replace YOUR_VALID_SESSION_ID_HERE and YOUR_OWN_USERNAME_HERE with actual values from your target application for the template to function correctly in an authenticated context. Manual analysis of API responses is crucial to identify appropriate regex patterns for extraction and unique identifiers for matching.

Integrating Recon for Enhanced Coverage

Nuclei's power is amplified when integrated into a broader reconnaissance workflow. Feeding a rich list of potential targets and endpoints into Nuclei ensures maximum coverage. Tools from ProjectDiscovery are designed to work together seamlessly in a pipeline.

• subfinder: Enumerates subdomains for a given target.
• httpx: Probes discovered subdomains for live HTTP/S services, filters out dead hosts, and extracts HTTP metadata.
• katana: A fast and customizable crawler that discovers URLs and endpoints from live hosts.

#!/bin/bash

# Configuration
TARGET_DOMAIN="example.com"
OUTPUT_DIR="recon_$(date +%Y%m%d_%H%M%S)"
CUSTOM_TEMPLATES_PATH="./custom-nuclei-templates" # Path to your custom templates

echo "[+] Initializing reconnaissance for $TARGET_DOMAIN..."
mkdir -p "$OUTPUT_DIR"

# Step 1: Subdomain Enumeration
echo "[+] Running subfinder for subdomains..."
subfinder -d "$TARGET_DOMAIN" -silent | sort -u > "$OUTPUT_DIR/subdomains.txt"
if [ ! -s "$OUTPUT_DIR/subdomains.txt" ]; then
    echo "[-] No subdomains found. Exiting."
    exit 1
fi
echo "[+] Found $(wc -l < "$OUTPUT_DIR/subdomains.txt") subdomains."

# Step 2: HTTP/S Probe
echo "[+] Probing live HTTP/S hosts with httpx..."
cat "$OUTPUT_DIR/subdomains.txt" | httpx -silent -mc 200,302,401,403 -retries 2 -fr -o "$OUTPUT_DIR/live_hosts.txt"
if [ ! -s "$OUTPUT_DIR/live_hosts.txt" ]; then
    echo "[-] No live HTTP/S hosts found. Exiting."
    exit 1
fi
echo "[+] Found $(wc -l < "$OUTPUT_DIR/live_hosts.txt") live hosts."

# Step 3: Endpoint Crawling
echo "[+] Crawling live hosts with Katana for endpoints..."
katana -list "$OUTPUT_DIR/live_hosts.txt" -dr -jc -kf -rl 50 -d 3 -ef png,jpg,gif,css,js,woff,ttf,svg -silent -o "$OUTPUT_DIR/endpoints.txt"
if [ ! -s "$OUTPUT_DIR/endpoints.txt" ]; then
    echo "[-] No endpoints crawled. Proceeding with hosts for direct API checks."
    # If no endpoints from katana, use live_hosts for nuclei
    cp "$OUTPUT_DIR/live_hosts.txt" "$OUTPUT_DIR/targets_for_nuclei.txt"
else
    echo "[+] Found $(wc -l < "$OUTPUT_DIR/endpoints.txt") endpoints."
    cp "$OUTPUT_DIR/endpoints.txt" "$OUTPUT_DIR/targets_for_nuclei.txt"
fi

# Step 4: Custom Nuclei Scan
echo "[+] Running custom Nuclei templates for Shadow API and BOLA detection..."
echo "    Using templates from: $CUSTOM_TEMPLATES_PATH"
# Ensure your custom templates are in the specified path, e.g., custom-nuclei-templates/shadow-api-common-paths.yaml
# and custom-nuclei-templates/bola-chained-id-manipulation.yaml
nuclei -l "$OUTPUT_DIR/targets_for_nuclei.txt" \
       -t "$CUSTOM_TEMPLATES_PATH/shadow-api-common-paths.yaml" \
       -t "$CUSTOM_TEMPLATES_PATH/bola-chained-id-manipulation.yaml" \
       -s high,medium,info -es info -o "$OUTPUT_DIR/nuclei_results.txt" \
       -silent -stats

echo "[+] Scan complete. Results in $OUTPUT_DIR/nuclei_results.txt"
echo "[+] Full output directory: $OUTPUT_DIR/"

# Example Nuclei command output (simulated)
echo -e "\n--- Nuclei Scan Example Output ---"
echo "[INF] Total requests: 120"
echo "[INF] Exhausted targets: 5"
echo "[INF] Templates completed: 2"
echo "[bola-chained-id-manipulation] [high] https://api.example.com/api/v1/users/789"
echo "[shadow-api-common-paths] [info] https://dev.example.com/api/v2/health"
echo "[bola-chained-id-manipulation] [high] https://test.example.com/api/v1/users/456"
echo "[INF] Scan took 0m25s"

This Bash script orchestrates the tools. It first finds subdomains with subfinder, then filters for live HTTP/S hosts with httpx, crawls those hosts for endpoints using katana, and finally feeds the resulting URLs to Nuclei. Nuclei then runs your custom Shadow API and BOLA templates against this target list. The -t flag specifies individual template paths, and -s filters by severity, while -es info excludes informational findings from the final output, focusing on more critical issues.

Practical Template Construction: Anatomy of a BOLA Template

Understanding the core components of a Nuclei template is essential for effective custom vulnerability detection.

• id: A unique identifier for the template.
• info: Contains metadata like name, author, severity, description, and tags. Tags are crucial for filtering scans.
• http: This block defines the HTTP requests to be sent.
  • raw: Allows full control over the HTTP request, including method, path, headers, and body. This is vital for complex API interactions.
  • Cookie: For authenticated tests, session cookies or authorization tokens must be included.
  • extractors: Captures data from a response (e.g., an ID, a token) using regex, JSONPath, or XPath. The internal: true flag makes the extracted value available to subsequent requests within the same template or workflow.
  • variables: Defines static or dynamically generated variables within the template.
  • matchers: Defines the conditions under which a request is considered a hit. Types include status (HTTP status codes), word (substrings in response), regex (regular expressions), and dsl (Domain Specific Language for complex logic). matchers-condition: and or or specifies how multiple matchers are combined.

For BOLA, dynamic ID manipulation is key. This often involves chaining requests where a first request, possibly authenticated as user A, retrieves a list containing user B's ID. Then, a second request, still authenticated as user A, attempts to access user B's resource using the extracted ID.

Automation Workflow: From Discovery to Vulnerability

The described workflow provides a robust foundation for automated Shadow API and BOLA detection. This process shifts left, integrating security checks earlier in the development lifecycle and continuously monitoring for new exposures.

To summarize the typical execution:

1. **Target Acquisition**: Define target domains or IP ranges.
2. **Reconnaissance**: Use tools like subfinder to enumerate subdomains, httpx to identify live web servers, and katana to crawl for all possible URLs.
3. **Endpoint Filtering**: Filter the crawled URLs for those likely to be API endpoints (e.g., containing /api/, /v1/, or specific keywords).
4. **Custom Nuclei Scan**: Run Nuclei with your bespoke Shadow API and BOLA templates against the filtered list of URLs.
5. **Analysis & Reporting**: Review Nuclei's output for identified vulnerabilities. Integrate with reporting tools as needed.

The power of custom Nuclei templates lies in their adaptability. As new API patterns emerge or specific business logic flaws are identified, new templates can be rapidly developed and deployed. This agile approach to vulnerability detection significantly enhances an organization's security posture against pervasive API threats.