Skip to content
Smart Data

A research product built on data from 40,000 sources

Smart Data worked alongside a legal and business information provider's product team to build a new, revenue-generating research product that loads, tags and stages data on millions of companies and executives.

Industry
Information services
Engagement
Data Platform
People at a window overlooking the City of London skyline

A large electronic publisher and data aggregator, known for legal research, business research and risk management services, wanted a new online product. Its corporate customers needed to research companies, industries and people, with up-to-date results from a single search.

SDS, a Smart Data company, worked closely with the client's product development team to take the idea and the existing data sources and turn them into a robust, easy-to-navigate product that generates revenue.

The challenge

  • Millions of company and executive profiles from around the world
  • Data aggregated from 40,000 news, business and public data sources
  • Everything had to be stored in a searchable format and kept up to date
  • The search interface had to stay easy and intuitive on top of that volume

The solution

  • An automated system that loads and updates large volumes of data on companies, industries and people
  • Incoming files are processed to decide which data is relevant, extract it and tag it with metadata
  • Attributes are stored for research and retrieval, keeping the relationships between thousands of data attributes
  • A staging database updates the online repository, so search and retrieval stay fast
  • A front end that reads from the staged data

The results

  • A new, revenue-generating online research product

  • Reports with a streamlined table of contents linking to company overviews, news on target companies, parent and subsidiary details, financial, competitive and stock information, recent mergers and acquisitions, legal information, and domestic and foreign patents

  • Financial overviews of companies worldwide, available from the same search

Where this is going

We build the data pipelines behind products that depend on large, constantly changing data sets. If that is your situation, talk to us about scope.

Client

Industry

Technology

Platform and technology

  • ETL & Data Pipelines
  • Data Modeling

Related services

Data engineering

Working on something like this?

Talk to us about scope

Working on something similar?

Tell us what you are building and what it has to connect to. Here is what happens after you reach out.

  1. A short call about the work

    What runs today, what needs to change and the date you are working toward.

  2. An engineer looks at what’s running

    Scope comes from your systems as they are, before anything is priced.

  3. You get a written proposal

    Covering scope, team, approach and price.

info@smartdata.net(937) 886-9166

Tell us what you are working on

We use your details only to reply.

Your privacy choices

Choose what this browser shares with us. The site works the same either way. Privacy policy