AFAYNOU M.

Data Engineer

415 dollar
Freelancer
2 years
Paris, FRANCE

My experience

More

MobiliseJanuary 2020 - Present

 1- Gathering Requirement from customer
2- Using AWS DMS to import data from various data source and store it in a perticular data format in AWS S3
3- Using AWS DMS to do an incremental load

4- Using AWS lambda to create table in glue catalog using
AWS athena by triggering a AWS lambda function
5- Dynamically creating the table using properties file in glue catalog
6- Using AWS CloudWatch to check the logs for data
7- Partition the table in AWS glue to reduce the cost while querying from Atena
8- Following best practices to reduce the cost from client

prospective
9- Using Pyspark to process data using AWS glue
10- Using AWS Redshift to store the final processed data
More

Mobilise Cloud · Contract Part-timeOctober 2020 - November 2021

Mission: RATP Group

AWS stacks( DMS, S3, Glue, Althena, Lambda, Redshift,...)

More

DeezerApril 2020 - August 2020

 Building and automatization of the Batch Data Pipeline
(Data Warehouse) using AWS cloud services (S3, EC2, Redshift, etc.) and language Python ( pandas,

pyspark, psycopg2, boto3, etc)

PROFESSIONAL ACADEMIC

M2 Data Science 'D2SN': parcours Big Data

My stack

Technologies

AWS

Environment of Development

GitLab

Languages

Python, SQL

IT Infrastructure

Docker, Azure, Amazon Redshift, Git, Cloud Computing

Analysis methods and tools

DevOps

Big Data

Big Data, Hadoop, Spark

Other

Data Warehousing, Amazon Web Services, Apache Cassandra

Databases

Cassandra, NoSQL, PostGreSQL

My education and trainings

M2 Data Science 'D2SN': parcours Big Data - UGE2020 - 2021

M1 Applied Mathematics: parcours Mathematics of Modelling - UPMC2019 - 2020