Deduplication

Remove near-duplicate records from a JSON array with fuzzy matching on the fields you choose, at the threshold you set.

Pay by wallet (x402) deduplicate

Example

Example answer

{
  "threshold": 0.8,
  "deduplicated": [
    {
      "age": 30,
      "city": "Stockholm",
      "name": "Alice"
    },
    {
      "age": 25,
      "city": "Oslo",
      "name": "Bob"
    }
  ],
  "match_fields": [
    "age",
    "city",
    "name"
  ],
  "removed_count": 0,
  "original_count": 2,
  "duplicates_found": [],
  "deduplicated_count": 2
}

Inputs

Field	Type	Required	What it is
`data`	array	Yes	Array of objects to deduplicate
`threshold`	number	No	Match threshold 0-1 (default 0.8)
`match_fields`	array	No	Fields to compare

Output fields

How reliably each field appears in a successful answer.

Field	Type	Appears
`deduplicated`	array	Always (present in every successful answer)
`original_count`	integer	Always (present in every successful answer)
`duplicates_found`	array	Always (present in every successful answer)
`deduplicated_count`	integer	Always (present in every successful answer)

How to call it

The same tool through each route. MCP clients reach it through Strale’s server at https://api.strale.io/mcp; the call below is what the agent sends.

# Needs an API key. max_price_cents caps what this call may cost.
curl -X POST https://api.strale.io/v1/do \
  -H "Authorization: Bearer $STRALE_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "capability_slug": "deduplicate",
    "inputs": {
      "data": [
        {
          "age": 30,
          "city": "Stockholm",
          "name": "Alice"
        },
        {
          "age": 25,
          "city": "Oslo",
          "name": "Bob"
        }
      ]
    },
    "max_price_cents": 2
  }'

{
  "tool": "strale_execute",
  "arguments": {
    "slug": "deduplicate",
    "inputs": {
      "data": [
        {
          "age": 30,
          "city": "Stockholm",
          "name": "Alice"
        },
        {
          "age": 25,
          "city": "Oslo",
          "name": "Bob"
        }
      ]
    },
    "max_price_cents": 2
  }
}

import { Strale } from "straleio";

const strale = new Strale({ apiKey: process.env.STRALE_API_KEY! });
const result = await strale.do({
  capability_slug: "deduplicate",
  inputs: {
    "data": [
      {
        "age": 30,
        "city": "Stockholm",
        "name": "Alice"
      },
      {
        "age": 25,
        "city": "Oslo",
        "name": "Bob"
      }
    ]
  },
  max_price_cents: 2,
});

import os
from straleio import Strale, DoRequest

strale = Strale(api_key=os.environ["STRALE_API_KEY"])
result = strale.do(DoRequest(
    capability_slug="deduplicate",
    inputs={
        "data": [
            {
                "age": 30,
                "city": "Stockholm",
                "name": "Alice",
            },
            {
                "age": 25,
                "city": "Oslo",
                "name": "Bob",
            },
        ],
    },
    max_price_cents=2,
))

# No account. The first request answers 402 with the price in USDC;
# an x402 client pays it from your wallet and repeats the request.
curl -i -X POST https://api.strale.io/x402/v2/deduplicate \
  -H 'Content-Type: application/json' \
  -d '{
    "data": [
      {
        "age": 30,
        "city": "Stockholm",
        "name": "Alice"
      },
      {
        "age": 25,
        "city": "Oslo",
        "name": "Bob"
      }
    ]
  }'

Limitations

Structurally different duplicates can be missed
Uses exact and fuzzy matching — records that are semantically duplicates but structurally different may not be caught
What to do instead: Adjust the similarity threshold parameter and review the flagged duplicates before automatic removal