We’re Hiring! | View Open Career Opportunities –>
We’re Hiring! | View Open Career Opportunities –>
The end-to-end platform to work with unstructured data. Get live in minutes with pre-built components, or run custom pipelines on our fully managed infrastructure to save months of engineering time.
We offer fully-managed connectors from industry leading providers. Tap into user-generated content at scale with agile data procurement.
Data sources that are ready out-of-the-box. Simply select them in your queries.
Fully managed connectors that only require your API key/token to use.
Pull internal data through an API or by dragging & dropping documents.
Say goodbye to manual data indexing and laborious schema unification. Free engineers from data plumbing to speed up product development.
Transform unstructured data to be:
Example Pipeline – Consumer Insights: This data pipeline feeds enriched user-generated content into a consumer insights product used by Fortune 500 companies.
Our tool is not just about ingesting vast quantities of data, it’s about making sense of it. Filter out the noise and enrich data with built-in AI models developed by world class teams.
Example Pipeline: Search Twitter & Instagram for posts with the word “cats”, and then enrich results with sentiment & intent classifiers.
curl --location 'https://api.platform.datastreamer.io/api/search' \
--header 'apikey: xxx' \
--header 'Content-Type: application/json' \
--data '{
"query": {
"query": "cats",
"data_sources": [
"wsl_twitter",
"wsl_instagram"
],
},
"operations": [
{
"name": "sentiment",
"destination_path": "operations.sentiment",
"parameters": {
"language": "enrichment.language",
"main": "content.body"
}
},
{
"name": "intent",
"destination_path": "operations.intent",
"parameters": {
"language": "enrichment.language",
"main": "content.body"
}
}
{
"results": [
{
"id": "1681502878992200182-artemis",
"data_source": "wsl_twitter",
"source": {
"link": "https://twitter.com/Uncanny_Vanny/status/xxxx"
},
"content": {
"body": "Honeslty love the idea that William is a cat person. Like yeah he loves bunny themed stuff but cats would be a close second.",
"found": "2023-04-14T20:07:58Z",
"published": "2023-04-14T20:07:15Z",
"favorites": 0,
"followers": 163,
"following": 0
},
"author": {
"name": "🐇🔪💜 VANNY 💜🔪🐇 #FireLadyFiszi",
"bio": "🐇🔪💜 -VANNY- 💜🔪🐇",
"location": "Freddy's Mega Pizzaplex",
"profile_image_source": "https://pbs.twimg.com/profile_images/xxxx/fbvJisCU_normal.jpg",
"gender": "UNKNOWN",
"url": "https://twitter.com/xxxx",
"handle": "xxxx"
},
"enrichment": {
"language": "en",
"sentiment": "POSITIVE",
]
},
"twitter": {
"tweet_type": "POST",
"retweet_type": "QUOTE",
"post_identifier": "1646968378366763009",
"user_verified": false,
"user_id": "1542407216115830784"
},
"operations": {
"sentiment": {
"label": "positive",
"confidence": 0.9996
},
"intent": {
"label": "yes",
"confidence": 0.9889
}
]
}
}
}
],
"total": {
"value": 10000,
"relation": "gte"
}
}
No more compatibility issues with complex data – just swift and simple integration into your warehouses or infused into your products with zero hassle.
Enter a world where your data is not just stored, but optimally organized and ready for action. Our data lakes keep your data conveniently accessible for robust data pipelines.
Find the needle in the haystack without drowning in complexity. Filter data with simple queries to boost analytical productivity.
Cater to your customers’ need for rapid insights from fluctuating data sources. Expand your coverage to billions of documents without the strenuous work.