Over 35 years of entertainment

Optimising big data queries for large volumes of complex data

22 July 2022 | Noor Khan

Optimising big data queries for large volumes of complex data in Amazon Athena

Key Challenges

Our client's existing system was unable to handle the large volumes of data they were dealing with on a daily basis and required a flexible, robust data reporting engine for time-efficient results.

Key Details

Service

Data Engineering

Technology

Cloud Watch, Cloud Trail, IAM, DynamoDB, SQL, Athena, S3 Data Lake, Angular, API Gateway, Lambda, AWS Batch

Industry

Media

Sector

Broadcasting

Key results

  • Ability to export a large amount of data to Excel
  • Multi-factor authentication to validate users for security purposes
  • Ability to save preferences and default settings for searches increasing efficiency when carrying out future searches
  • Speedy and accurate data results
  • Email alerts to inform users when data results are ready so they do not have to wait around
  • High level of flexibility when carrying out searches with capabilities of combination searches

Multinational broadcasting company

Entertaining and inspiring audiences since 1984

Our client are a well-established, multi-national broadcasting company reaching 300 million subscribers across 150 countries, through 50 channels and in 37 languages. They deal with large sets of complex data audience data ranging from streaming to TV audiences. They need to be able to efficiently carry out searches, for research and sales purposes. Their data sets are sourced from Nielsen AMRLD which is stored in their data warehouse.

Optimising big data queries for large volumes of complex data

Program Search and Market Break Ranker tool

Huge data requires a powerful system

Our client’s existing data search tool performed the search queries extremely slow, frequently timed out because of the large volumes of complex data and did not offer the flexibility required. They needed a tool that would help them gain insights from large amounts of data to help support the research and sales efforts of the business.

Optimising big data queries for large volumes of complex data

Flexible, user-friendly tool that handles data searches on large volumes of complex data

Making big data searches, speedy

The client was looking for an improved data search tool that allowed them to have a Program Search as well as a Market Break Ranker. These two different components were essentially search engines for the two types of data the company dealt with .The Ardent engineering team took on board the requirements and created a tool that optimised big data queries for large volumes of complex data with a user-friendly interface, using API Gateway and Angular technologies. We offered flexibility by adding filters and drop-down menus, allowing the users to select multiple categories for the data results see t.As the client wanted the ability to be able to carry out Boolean logic combination searches, we needed to build a system that could search vast data sets to pull out relevant data results. We were able to provide a solution on Lambda and then moved to AWS Batch to ensure a quick and efficient search of the data.

More on our technology partners.

Optimising big data queries for large volumes of complex data

Ardent's data engineering expertise

Drill down searches with boolean logic

Our expert engineers made recommendations on boolean logic to streamline data searches as we have a vast amount of experience working on a similar scale of data and clients.

Optimised big data queries

Searching huge data sets quickly and easily

Our experienced engineers delivered a fit for purpose data solution, allowing our client to optimise big data queries on large, complex big data with a number of features to make the search process easy and straightforward. This enabled our client to have a quick turnaround time on analysis of the data (household viewing figures), increasing efficiency in the data search process. Overall, they were able to save time, be efficient, and save costs allowing them to focus on other important business goals.

Explore our data engineering services or get in touch to find out how we can help you optimise your big data.


More Success Stories

Ensuring timely data availability for real time mission critical data

Success Story

Monetizing broadcasting data

Media | Media

A market leader, internationally renowned media and broadcasting company Founded in 2002, our client has been around for over two decades and is an internationally known company dealing with broadcasting data for commercial use. With a mission of making high-quality technology and content affordable for everyone, they have established themselves as a market leader. [...]

Read More... from Over 35 years of entertainment

Fine art storage & preservation software

Success Story

Making logistics simple

Logistics | Logistics, Software

Leader logistics software provider Our client is a leading logistics software provider in the UK. With over 3 decades of experience in the industry, they continuously look to innovate with technology. Their range of software products includes a warehouse management system and removal management software. They aim to remove the complexity of software and bring [...]

Read More... from Over 35 years of entertainment

warehouse management automation user-friendly app

Success Story

Three decades of experience in delivering software excellence

Technology | Logistics, Software

Well-established logistics software provider Our client is a software products company providing software to the logistics industry and their main product was administration solution software for removal companies. With almost three decades of experience, our clients are leaders in the removals sector. Since the start, they have gone from strength to strength in becoming a [...]

Read More... from Over 35 years of entertainment

Ardent Insights

Are you ready to take the lead in driving digital transformation?

Are you ready to take the lead in driving digital transformation?

Digital transformation is the process of modernizing and digitating business processes with technology that can offer a plethora of benefits including reducing long-term costs, improving productivity and streamlining processes. Despite the benefits, research by McKinsey & Company has found that around 70% of digital transformation projects fail, largely down to employee resistance. If you are [...]

Read More... from Over 35 years of entertainment

Stateful vs Stateless

Stateful VS Stateless – What’s right for your application?

Protocols and guidelines are at the heart of data engineering and application development, and the data which is sent using network protocols is broadly divided into stateful vs stateless structures – these rules govern how the data has been formatted, how it sent, and how it is received by other devices (such as endpoints, routers, [...]

Read More... from Over 35 years of entertainment

Getting data observability done right - Is Monte Carlo the tool for you (1)

Getting data observability done right – Is Monte Carlo the tool for you?

Data observability is all about the ability to understand, diagnose, and manage the health of your data across multiple tools and throughout the entire lifecycle of the data. Ensuring that you have the right operational monitoring and support to provide 24/7 peace of mind is critical to building and growing your company. [...]

Read More... from Over 35 years of entertainment