Lead Data Engineer

€ Very competitive plus benefits

Permanent
Dublin City, Leinster
Posted 3 weeks ago

Our Client, SmartFlow Compliance Solutions, is a leading developer of anti-piracy and license compliance technology. They are an innovative start-up company that has already established itself as the preferred supplier to industry leading enterprise software companies.  With comprehensive software telemetry & SaaS based business intelligence solutions, their technology allows customers to track unlicensed corporate usage of their valuable software, and convert unlicensed corporate users into paying customers.

SmartFlow are now looking to recruit a Lead Data Engineer in Dublin with a minimum of 8 years’ relevant development experience (Python or Scala) building scalable enterprise solutions in an AWS cloud environment, and with significant Experience of big data tools such as Apache Kafka, Elastic Search, Redis.

Core Role Responsibilities:

  • Design and develop a cloud based enterprise solution to connect multiple data feeds and repositories as part of a scalable data processing solution.
  • Lead a small team of data engineers to build production quality data processing solutions.
  • Provide guidance to junior members of the team on technology choices, design and implementation of data integrations.
  • Continuous re-evaluation of existing data processing solutions to maintain optimal performance, data security and availability.
  • Assure data quality using data cleansing techniques.
  • System troubleshooting and problem solving.
  • Documenting the configuration and operation of the data integration platform.
  • Evaluation and adoption of suitable new technologies, for example No SQL and Graph Databases, Big Data tools.
  • Other activities and responsibilities that may be deemed appropriate as required.

Required Experience and Skills:

  • AWS Cloud: Experience developing/deploying in an AWS Cloud environment.
  • Big Data tools: Significant Experience of big data tools such as Apache Kafka, Elastic Search, Redis.
  • Scripting Skills: Strong scripting and automation skills (using Python).
  • Data Interchange: Developing with XML, JSON based technologies.
  • Web Services: Developing and integrating web services (ReST, SOAP).
  • Deployment Containerization: Knowledge of Docker.

 Desirable Skills (Not essential but beneficial to have):

  • Relational Database Skills: Experience writing SQL for data ETL.
  • Agile Methodologies: Experience working in an Agile development environment.
  • Monitoring Tools: Experience in integration of monitoring systems (Zabbix, CloudWatch, Kibana) with ETL services.

Educational Requirements:

  • BSc in Computer Science or other equivalent technical discipline.

Job Features

Job CategoryInformation Technology
Salary€ Very competitive plus benefits

Apply Online

A valid email address is required.