Apply for the job

Data Engineer/Senior Data Engineer

Samya ai

Samya ai

Bengaluru, Karnataka, IN.

FULL TIME

Category

Computer and Mathematical Occupations

Published on 2022-07-27 16:33

About Samya.AI


We are an AI/ML cloud-based SaaS product company. We use AI and Machine Learning to reduce operational blind spots, anticipate risks, and close the gap between planning and execution.

Samya helps organizations unlock revenue growth potential at the intersection of demand and supply.


The company was created with a vision to build a world-class product and team. Our team combines strengths from various disciplines, including Product Development, R&D, AI/ML, and Engineering teams.


Samya is a term that means "bringing things to equilibrium". In an environment of increasing complexity and uncertainty, AI and machine learning can help organizations achieve dynamic

equilibrium and maximize growth.


What We are looking for :


Seeking individuals with demonstrated experience in data engineering for developing and maintaining data engineering modules and pipelines for our SAAS offering. Your primary focus will be to develop/enhance/optimize data engineering pipelines. You will also be responsible for supporting customer’s proof of validations and implementations from data engineering perspectives


Skills required:


  • Experience of working on public clouds (AWS / Azure)
  • Good Coding skills in Python, Pyspark and SQL is must.
  • Experience of writing/maintaining Spark code base in a production environment.
  • Big Data Experience - Spark, Relational Databases (Postgres, etc.)
  • Worked with orchestration technologies like Airflow, Kubeflow etc.
  • Build & support scalable data engineering pipelines (ETLs etc.). This will entail extract, load and transform of ‘big data’ from a wide variety of sources, both batch & streaming, using latest data frameworks and technologies, along with real time monitoring dashboards and alerting.
  • Essential experience of working with distributed systems software development.
  • Demonstrated experience of production experience in big data infrastructure and data modelling.
  • Critical experience performance optimization for both data loading and data ingestion.
  • Know-how of critical tools, in a developer’s toolkit such as GitHub, Dockers, VSCode
  • Ability to work in a fast-paced and deadline driven environment.


Responsibilities:


  • Build & support scalable data engineering pipelines (load, extract, cleaning, transform, ingest etc.) for our SaaS AI Applications and Predictions-as-a-Service Implementations and Product.
  • Build and Deploy the Data Engineering pipelines for scale using Kubeflow
  • Have ownership of multiple data pipelines.
  • Review, audit and improve the Data Engineering pipelines
  • Document and communicate technical and functional design (HLD, LLD), timelines and plans
  • Work closely with AI/ML, DevOps, MLE, other engineering and product team
  • Work with the engineering and implementation team to understand business problems and implement efficient software solutions
  • Follow configurable programming practices (config files, modular programming, parameterization, etc.)
  • Support Maintenance/enhancement of deployed pipelines. This includes bug-fixes and performance improvement.
  • Participate in organization building activities (cross team training, hiring, etc.) to further the Samya mission and culture
© 2022 JobSearchine India