AWS Data Engineer

  • Database
  • Remote
  • Permanent / Full Time

About the role 
Job Title: AWS Data Engineer 
Location: Remote across India
Job Type: Permanent  


Job Description
Must Have

  • AWS Services Expertise: Advanced knowledge of AWS services such as DynamoDB (designing and managing tables), S3 (data ingestion and transformation), IAM (roles, policies, permissions), Glue (ETL jobs, Crawlers), and security controls (encryption, key management).
  • ETL and Data Pipelines: Strong understanding of ETL processes, various data formats (CSV, Parquet, JSON), SQL for data transformation, and creating scalable, optimized data pipelines with AWS Glue and PySpark.
  • Workflow Orchestration: Experience with tools like StepFunction, Apache Airflow, EventBridge, and Lambda for setting up schedules or event triggers for pipeline execution.
  • Pipeline Optimization: Knowledge of optimizing data pipelines for performance, scalability, and cost-effectiveness, including partitioning, indexing, and parallel processing.
Nice to Have
  • API Development: Proficiency in RESTful API development using Python and/or Java, including API security (OAuth, JWT) and testing with tools like Postman.
  • Logging and Monitoring: Skills in implementing job execution logging, error handling, and monitoring using AWS CloudWatch Logs and alerting mechanisms.
  • Data Access and Compliance: Proficiency in cross-account access management, audit trails, logging mechanisms, and adherence to industry compliance standards
 
 

Apply Now