Frisma R.

Data Engineer

530 dollar
Freelancer
3 years
London, UNITED KINGDOM

My experience

More

AccentureJanuary 2019 - Present

Project: Develop end to end DataOps solution with ETL framework using PySpark/Scala and build integrated ETL pipelines for data processing orchestrated through DevOps Containerized platform built on Kubernetes

 Roles & Responsibilities:

• Developed data pre-processing module for batch and streaming data across multiple data sources using Pyspark

• Implemented test and simulation framework for automated testing of data transformation module and wrote unit test cases of ETL application

• Developed parameterized Jenkins pipeline using groovy script.

• Implemented Dask multiprocessing with dynamic spark clusters to invoke Dask executors.

• Developed web based applications using python

• Played key role in implementing EDA (Explo- ratory data analysis), data quality, data cleaning and data validation using Python scripting.

• Developed data logging and lineage monitoring programs using Python, Java, shell scripting to push logs to data analytics solution.

• Implemented API approach to package application with Flask API

• Created standard operating procedure documentation to assist SDLC process and user documentation for easy implementation of solution.

More

BTC techno labs cochinSeptember 2017 - April 2018

Project: Develop fingerprint valuation algorithm and implement voice-based system for blinds using google voice modules. 

 Roles & Responsibilities:

• Developed fingerprint valuation algorithm using Python

• Developed python packages to handle user level exceptions 

• Developed stored procedures and functions on Oracle database

• Web based development experience utilizing python.

My stack

Languages

HTML, JavaScript, Java, Scala, Python, SQL

Analysis methods and tools

DevOps

Business Intelligence

ETL

Databases

Microsoft SQL Server, Oracle

Technologies

HDFS, AWS, Pandas

Big Data

Big Data, PySpark

Middleware

Jenkins

Open Source solutions

Kubernetes

IT Infrastructure

Amazon Identity and Access Management (IAM), Azure DevOps, Amazon EC2

Others

Data processing, Analytics, API, SDLC

Software testing

Test Cases

My education and trainings

AWS Certified Developer – Associate - -2021 - 2024

Bachelor of Technology, Computer science - School of Engineering2014 - 2017

Diploma, Computer Engineering - Government Polytechnic College2011 - 2014