Web Fetch - profClaw

Overview

web_fetch lets the agent make outbound HTTP requests. It supports all common HTTP methods, custom headers, request bodies, and can optionally extract readable text from HTML pages. Every request passes through SsrfGuard before execution - a defense layer that blocks private IP ranges, cloud metadata endpoints, and DNS rebinding attacks.

Tool: `web_fetch`

Security level: moderate | Tier: Essential

string

required

Full URL to fetch. Must be a valid http:// or https:// URL.

string

default:"GET"

HTTP method: GET, POST, PUT, DELETE.

object

Custom request headers as key-value pairs.

string

Request body for POST or PUT. Typically JSON-encoded.

number

default:"30"

Request timeout in seconds. Maximum enforced by server policy.

boolean

default:"false"

Extract readable text content from HTML pages (strips tags, navigation, scripts).

Examples

{
  "url": "https://docs.example.com/api",
  "extractText": true
}

{
  "url": "https://api.github.com/repos/profclaw/profclaw",
  "headers": {
    "Accept": "application/vnd.github.v3+json"
  }
}

{
  "url": "https://hooks.slack.com/services/T00/B00/xxx",
  "method": "POST",
  "headers": { "Content-Type": "application/json" },
  "body": "{\"text\": \"Build completed successfully\"}"
}

{
  "url": "https://api.example.com/v1/status",
  "headers": {
    "Authorization": "Bearer eyJhbGci..."
  }
}

Response

A successful fetch returns:

{
  "success": true,
  "data": {
    "url": "https://api.example.com/status",
    "status": 200,
    "contentType": "application/json",
    "body": "{ \"status\": \"ok\" }",
    "bodyLength": 16
  }
}

Content Limits

Responses are capped at 500KB. Larger responses are truncated. For large downloads, consider fetching a specific resource path rather than a whole page.

SSRF Protection

The SsrfGuard blocks requests to:

Category	Examples
Loopback	`127.0.0.1`, `::1`, `localhost`
Private networks	`10.x.x.x`, `172.16-31.x.x`, `192.168.x.x`
Link-local	`169.254.x.x` (includes cloud metadata at `169.254.169.254`)
Cloud metadata	`metadata.google.internal`, `metadata.internal`
Reserved ranges	RFC 1918 + all IANA special-use ranges

The guard resolves DNS before connecting to defend against DNS rebinding attacks. Redirect chains are re-validated at each hop, up to 5 redirects.

To allow specific internal hosts (e.g., for self-hosted integrations), add them to security.ssrfGuard.allowedHosts in your settings.yml.

Allowed Hosts Setting

security:
  ssrfGuard:
    enabled: true
    allowedHosts:
      - "internal-api.company.local"
      - "jenkins.internal"

Web Search

Search the web instead of fetching a specific URL.

Browser Tools

Interact with JavaScript-rendered pages using a real browser.

​Overview

​Tool: web_fetch

​Examples

​Response

​Content Limits

​SSRF Protection

​Allowed Hosts Setting

​Related Tools

Web Search

Browser Tools

Overview

Tool: `web_fetch`

Examples

Response

Content Limits

SSRF Protection

Allowed Hosts Setting

Related Tools