---
title: "Data Aggregation for a Research Product | Smart Data"
url: https://www.smartdata.net/case-studies/research-data-aggregation-information-provider
description: "Smart Data helped a legal and business information provider build a research product on data from 40,000 sources, with automated loading, tagging and staging."
source: "Smart Data (smartdata.net). Markdown copy of the page; the HTML page is canonical."
---

# A research product built on data from 40,000 sources

Smart Data worked alongside a legal and business information provider's product team to build a new, revenue-generating research product that loads, tags and stages data on millions of companies and executives.

Industry

Information services

Engagement

Data Platform

A large electronic publisher and data aggregator, known for legal research, business research and risk management services, wanted a new online product. Its corporate customers needed to research companies, industries and people, with up-to-date results from a single search.

SDS, a Smart Data company, worked closely with the client's product development team to take the idea and the existing data sources and turn them into a robust, easy-to-navigate product that generates revenue.

## The challenge

- Millions of company and executive profiles from around the world
- Data aggregated from 40,000 news, business and public data sources
- Everything had to be stored in a searchable format and kept up to date
- The search interface had to stay easy and intuitive on top of that volume

## The solution

- An automated system that loads and updates large volumes of data on companies, industries and people
- Incoming files are processed to decide which data is relevant, extract it and tag it with metadata
- Attributes are stored for research and retrieval, keeping the relationships between thousands of data attributes
- A staging database updates the online repository, so search and retrieval stay fast
- A front end that reads from the staged data

## The results

A new, revenue-generating online research product

Reports with a streamlined table of contents linking to company overviews, news on target companies, parent and subsidiary details, financial, competitive and stock information, recent mergers and acquisitions, legal information, and domestic and foreign patents

Financial overviews of companies worldwide, available from the same search

## Where this is going

We build the data pipelines behind products that depend on large, constantly changing data sets. If that is your situation, talk to us about scope.

Client

Technology

Platform and technology

- ETL & Data Pipelines
- Data Modeling

## Working on something similar?

Tell us what you are building and what it has to connect to. Here is what happens after you reach out.

### A short call about the work

What runs today, what needs to change and the date you are working toward.

### An engineer looks at what’s running

Scope comes from your systems as they are, before anything is priced.

### You get a written proposal

Covering scope, team, approach and price.

[info@smartdata.net](mailto:info@smartdata.net)[(937) 886-9166](tel:+19378869166)