As a Senior Data Engineer - Streaming Engineer', you will be responsible for designing, building, and configuring Big Data pipelines. You will work independently to develop high-performance data processing workflows and ensure the pipelines operate efficiently. Your role will also involve troubleshooting and resolving complex big data issues, ensuring smooth progress from development through to production and providing post-production support.
Contract Duration: 6+ months
Key Responsibilities:
1. Minimum of 1 year of IT experience, with at least 1 year of hands-on experience in Big Data implementation
2. Proven expertise in designing and building Big Data pipelines and systems
3. Experience with streaming data solutions, specifically using PySpark, Scala, and Kafka
4. In-depth experience with Spark (including Spark Streaming) to process large datasets in real-time
5. Strong understanding of database structures, principles, and practices related to Big Data
Technical Competencies:
1. Strong knowledge and experience with the Big Data stack, particularly PySpark, Scala, Kafka, and Spark Streaming
2. Proven ability to identify, troubleshoot, and resolve complex data issues within Big Data pipelines
3. Solid understanding of database structures, data management principles, and performance optimization
4. Familiarity with cloud-based environments (AWS, Azure, or GCP) is highly desirable
5. Knowledge of Agile development methodologies and their application in Big Data projects
Enterprise Competencies:
1. Ability to independently handle low-level design, development, and troubleshooting tasks
2. Excellent problem-solving skills with the ability to analyze and address complex technical challenges
3. Strong collaboration skills to work effectively with technical teams and business stakeholders
4. Strong communication skills to report progress, challenges, and solutions effectively
Additional Desired Skills:
1. Experience in cloud environments such as AWS, Azure, or GCP, particularly in a Big Data context
2. Familiarity with Agile methodology and iterative development processes
3. Ability to manage multiple tasks in a fast-paced environment while maintaining high-quality standards
Wrexa Technologies is a forward-thinking technology company dedicated to providing innovative solutions to meet the evolving needs of businesses across various industries. Founded by Dinesh Selvaraj, our company stands at the forefront of digital transformation, offering a comprehensive suite of services ranging from software development and digital strategy to IT consulting and technical support. At Wrexa Technologies, we pride ourselves on our commitment to excellence, creativity, and client satisfaction. Our team of experienced professionals combines technical expertise with a deep understanding of business dynamics to deliver tailor-made solutions that drive growth, efficiency, and competitive advantage for our clients.
