Overview
Website to Text works by parsing the HTML content of a website and extracting the text. Text extraction is done by removing the HTML tags and returning the plain text content. Advanced algorithms are used to ensure accurate text extraction.
Endpoint
One host, one path per API. The block below shows this call in four languages; every one of them is the same HTTP request. Making requests covers the timeouts, retries and parameter rules that apply to all of them. The SDKs wrap the same call in a typed client.
curl -X POST https://api.apiverve.com/v1/websitetotext \
-H "x-api-key: your_api_key_here" \
-H "Content-Type: application/json" \
-d '{
"url": "https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts"
}'const res = await fetch('https://api.apiverve.com/v1/websitetotext', {
method: 'POST',
headers: {
'x-api-key': 'your_api_key_here',
'Content-Type': 'application/json',
},
body: JSON.stringify({
"url": "https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts"
}),
});
if (!res.ok) throw new Error(`${res.status} ${await res.text()}`);
const { data } = await res.json();
console.log(data);import requests
res = requests.post(
"https://api.apiverve.com/v1/websitetotext",
headers={"x-api-key": "your_api_key_here"},
json={
"url": "https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts"
},
timeout=15,
)
res.raise_for_status()
print(res.json()["data"])package main
import (
"fmt"
"io"
"net/http"
"strings"
)
func main() {
body := strings.NewReader(`{
"url": "https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts"
}`)
req, _ := http.NewRequest("POST", "https://api.apiverve.com/v1/websitetotext", body)
req.Header.Set("Content-Type", "application/json")
req.Header.Set("x-api-key", "your_api_key_here")
res, err := http.DefaultClient.Do(req)
if err != nil {
panic(err)
}
defer res.Body.Close()
out, _ := io.ReadAll(res.Body)
fmt.Println(string(out))
}Replace your_api_key_here with the key from your dashboard. When the inputs arrive as a list rather than one at a time, batch requests run up to 200 of them through this same API in a single call.
Authentication
Send your key in the x-api-key header. That is the only auth step — there is no token exchange and no per-endpoint scope to configure. Authentication covers creating, rotating and revoking keys.
A 401 means the key is missing, invalid or expired. A 403 means the key is valid but not permitted here — blocked by a key restriction or an IP allow-list. Running out of credits is a 429.
Parameters
Sent as JSON in the request body. Premium parameters are accepted on every plan but only take effect on plans that include them.
| Parameter | Type | Description |
|---|---|---|
urlRequired | string | The URL of the website to convert to text url |
Response
Every API returns the same three top-level keys, so one response handler covers your whole integration: status, error and data. Only data changes shape. Response format covers the envelope, the other output formats and how premium fields are withheld.
{
"status": "ok",
"error": null,
"data": {
"date": null,
"description": "Use Amazon EC2 for scalable computing capacity in the AWS Cloud so you can develop and deploy applications without hardware constraints.",
"title": "What is Amazon EC2?",
"title_alt": "What is Amazon EC2?",
"text": "Amazon Elastic Compute Cloud (Amazon EC2) provides on-demand, scalable computing capacity in the Amazon Web \t\tServices (AWS) Cloud. Using Amazon EC2 reduces hardware costs so you can develop and deploy \t\tapplications faster. You can use Amazon EC2 to launch as many or as few virtual servers as you \t\tneed, configure security and networking, and manage storage. You can add capacity (scale up) \t\tto handle compute-heavy tasks, such as monthly or yearly processes, or spikes in website \t\ttraffic. When usage decreases, you can reduce capacity (scale down) again. An EC2 instance is a virtual server in the AWS Cloud. When you launch an EC2 instance, \tthe instance type that you specify determines the hardware available to your instance. \tEach instance type offers a different balance of compute, memory, network, and storage \tresources. For more information, see the Amazon EC2 Instance Types Guide. Amazon EC2 provides the following high-level features: Amazon EC2 supports the processing, storage, and transmission of credit card data by a merchant or service provider, and has been validated as being compliant with Payment Card Industry (PCI) Data Security Standard (DSS). For more information about PCI DSS, including how to request a copy of the AWS PCI Compliance Package, see PCI DSS Level 1. You can create and manage your Amazon EC2 instances using the following interfaces: Amazon EC2 provides the following pricing options: For a complete list of charges and prices for Amazon EC2 and more information about the purchase \t\t\tmodels, see Amazon EC2 pricing. To create estimates for your AWS use cases, use the AWS Pricing Calculator. To estimate the cost of transforming Microsoft workloads to a modern architecture that uses open source and \t\t\t\tcloud-native services deployed on AWS, use the AWS Modernization Calculator for Microsoft Workloads. To see your bill, go to the Billing and Cost Management Dashboard in the AWS Billing and Cost Management console. Your bill contains links to usage reports that provide details \t\t\t\tabout your bill. To learn more about AWS account billing, see AWS Billing and Cost Management User Guide. If you have questions concerning AWS billing, accounts, and events, contact AWS Support. To calculate the cost of a sample provisioned \t\t\t\t\tenvironment, see . When calculating the cost of a provisioned \t\t\t\tenvironment, remember to include incidental costs such as snapshot storage for EBS \t\t\t\tvolumes. You can optimize the cost, security, and performance of your AWS environment \t\t\t\tusing AWS Trusted Advisor. You can use AWS Cost Explorer to analyze the cost and usage of your EC2 instances. You can view \t\t\t\tdata up to the last 13 months, and forecast how much you are likely to spend for the next \t\t\t\t12 months. For more information, see \t\t\t\tAnalyzing your costs and usage with AWS Cost Explorer in the AWS Cost Management User Guide.",
"language": "en",
"publisher": null,
"url": "https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts"
}
}Response fields
Paths are relative to data. Premium fields are absent rather than zeroed on plans that do not include them, so check for presence instead of comparing to 0.
| Field | Type | Example | Description |
|---|---|---|---|
date | object | null | Publication or last modified date of the webpage |
description | string | "Use Amazon EC2 for scalable computing capacity in the AWS Cloud so you can develop and deploy applications without hardware constraints." | Meta description extracted from website content |
title | string | "What is Amazon EC2?" | Primary page title extracted from HTML |
title_alt | string | "What is Amazon EC2?" | Alternative title extracted from page content |
text | string | "Amazon Elastic Compute Cloud (Amazon EC2) provides on-demand, scalable computing capacity in the Amazon Web Services (AWS) Cloud. Using Amazon EC2 reduces hardware costs so you can develop and deploy applications faster. You can use Amazon EC2 to launch as many or as few virtual servers as you need, configure security and networking, and manage storage. You can add capacity (scale up) to handle compute-heavy tasks, such as monthly or yearly processes, or spikes in website traffic. When usage decreases, you can reduce capacity (scale down) again. An EC2 instance is a virtual server in the AWS Cloud. When you launch an EC2 instance, the instance type that you specify determines the hardware available to your instance. Each instance type offers a different balance of compute, memory, network, and storage resources. For more information, see the Amazon EC2 Instance Types Guide. Amazon EC2 provides the following high-level features: Amazon EC2 supports the processing, storage, and transmission of credit card data by a merchant or service provider, and has been validated as being compliant with Payment Card Industry (PCI) Data Security Standard (DSS). For more information about PCI DSS, including how to request a copy of the AWS PCI Compliance Package, see PCI DSS Level 1. You can create and manage your Amazon EC2 instances using the following interfaces: Amazon EC2 provides the following pricing options: For a complete list of charges and prices for Amazon EC2 and more information about the purchase models, see Amazon EC2 pricing. To create estimates for your AWS use cases, use the AWS Pricing Calculator. To estimate the cost of transforming Microsoft workloads to a modern architecture that uses open source and cloud-native services deployed on AWS, use the AWS Modernization Calculator for Microsoft Workloads. To see your bill, go to the Billing and Cost Management Dashboard in the AWS Billing and Cost Management console. Your bill contains links to usage reports that provide details about your bill. To learn more about AWS account billing, see AWS Billing and Cost Management User Guide. If you have questions concerning AWS billing, accounts, and events, contact AWS Support. To calculate the cost of a sample provisioned environment, see . When calculating the cost of a provisioned environment, remember to include incidental costs such as snapshot storage for EBS volumes. You can optimize the cost, security, and performance of your AWS environment using AWS Trusted Advisor. You can use AWS Cost Explorer to analyze the cost and usage of your EC2 instances. You can view data up to the last 13 months, and forecast how much you are likely to spend for the next 12 months. For more information, see Analyzing your costs and usage with AWS Cost Explorer in the AWS Cost Management User Guide." | Complete extracted text content from website |
language | string | "en" | Detected language code of website content (e.g. en) |
publisher | object | null | Publisher or source organization of webpage |
url | string | "https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts" | Original URL provided for text conversion |
Errors
Read the HTTP status first, then error for the specific reason. The body names the parameter that has to change. Error handling covers the full status list and which of them are worth retrying.
| Status | Meaning | What to do |
|---|---|---|
400 | Input was rejected | Read error; it names the parameter. |
401 | Key missing or invalid | Check the header name and the key value. |
403 | Key valid, but not permitted | A key restriction or IP allow-list; see key scoping. |
429 | Rate limited, or out of credits | Read error to tell them apart; see rate limits. |
Use cases
- Content Extraction
- Use the Website to Text API to extract content from a website. Use the data to scrape text, analyze content, and extract information
- Data Mining
- Convert websites to text by using the Website to Text API for data mining. Use the data to extract data, collect information, and analyze trends
- Text Analysis
- Analyze text by using the Website to Text API to convert websites to text. Use the data to study language patterns, identify themes, and classify content
Other ways to use Website to Text
Set up Website to Text on APIVerve, or reach the same source a different way. Your APIVerve account and credits work on all of them — one key, one balance.
Related
More in Data Conversion: