Data Architect (PySpark, Python, Spark, ETL)
Lahore, Punjab, Pakistan
Full Time
Data Engineering
Experienced
NorthBay Solutions, with a team of 400+ professionals, is a US-based premier AWS partner and the only premier-level partner in the region. We have been serving our clients globally and providing AWS Big Data, Data Lake & Data Warehousing solutions for over a decade now.
We are currently recruiting for a Tech Lead/Architect with solid experience in data engineering and a passion for Big Data. The ideal candidate will have experience of overall 12+ years and 5+ years specifically in data engineering.
Technology Stack Used & Required Knowledge:
We are currently recruiting for a Tech Lead/Architect with solid experience in data engineering and a passion for Big Data. The ideal candidate will have experience of overall 12+ years and 5+ years specifically in data engineering.
Technology Stack Used & Required Knowledge:
- Must have 12+ years experience of developing and implementing pipelines on AWS cloud that extract, transform, and load data into an information product that helps the organization reach its strategic goals
- Must have experience on ETL Data Engineer, Python, Bigdata (PySpark, Spark SQL, Hadoop, Hive) and/or Java
- Experience in creating and driving large-scale ETL pipelines in AWS-based environment
- Experience with integration of data from multiple data sources.
- Overall 5+ years of relevant work experience in Big data engineering, ETL, Data Modeling, and Data Architecture.
- Strong software development and programming skills with a focus on data using Python/ PySpark and/or Scala for data engineering.
- Experience and understanding of various core AWS services such as IAM, Cloud Formation, EC2, S3, EMR/Spark, Glue, Lambda, Athena, and Redshift will be a plus
- Experience with the AWS data management tools such as Data lake and Databricks or AWS Snowflake is a plus
- Design and develop using Big Data technologies.
- Design and develop cutting edge Big Data, and Cloud technologies to build critical, highly complex distributed systems from scratch
- Design, develop and manage complex ETL jobs and pipelines.
- Act as a critical thought leader, consulting and mentoring other teams as your new systems transform the way this entire firm leverages data
Apply for this position
Required*