Jagdish Wagh
Verified Expert in Engineering
Database Developer
Jagdish是一位在不同领域工作过的数据工程师, including retail, insurance, and manufacturing. 他在实现ETL管道和使用数据库和Kimball原理创建数据模型方面拥有丰富的经验. Jagdish曾在AWS和Azure云中工作,并使用AWS Redshift和Azure Synapse Analytics实现了一个云数据仓库.
Portfolio
Experience
Availability
Preferred Environment
Python, Microsoft Power BI, Data Modeling, Azure Data Factory, SQL, MSBI, Terraform, Databricks, ETL Tools, Data Warehouse Design
The most amazing...
...我用Kimball和data Vault 2开发了一个数据仓库和数据集市应用程序.0 principles using on-prem and cloud environments.
Work Experience
Data Engineer
medi GmbH & Co. KG
- 使用本地和云技术迁移和开发数据分析和数据仓库,并实现混合架构. 负责不同的用例、库存控制、生产订单和销售管理座舱.
- 创建了用于监视数据管道执行时间和错误消息的作业审计框架. 由于这个过程,我们能够尽快解决问题,节省了很多时间.
- 在自动化中使用PowerShell脚本实现了一个Azure分析多维数据集备份和恢复过程. 由于这个过程,我们可以在一分钟内恢复旧的多维数据集状态.
- 创建了一个人力资源管理数据集市,用于维护整个组织的工作流. Because of this dashboard, the whole management can see sickness percentage, headcounts, and paternity leave, and they were able to make decisions.
Senior Software Engineer
TietoEVRY
- Created a data model and enterprise data warehouse from scratch using the Kimball approach and handed it over to the team; built the ETL data pipeline using Databricks, Azure Data Factory.
- 使用PowerShell和Unix shell脚本创建自动化,使用AzCopy实用程序将数据文件从本地数据中心复制到Azure存储.
- 参与使用matilion和Informatica云ETL工具构建云数据仓库.
- 在Informatica和Azure Data Factory的帮助下,作为从本地Teradata迁移到Snowflake数据库和Azure Synapse Analytics的项目的一部分,在多个poc上工作.
- 使用Terraform和ARM模板在现有平台中集成了新的Azure网络服务.
Software Engineer
TietoEVRY
- 与客户一起了解业务需求,并在QlikView和Power BI中将这些业务需求转化为可操作的报告, saving 17 hours of manual work each week.
- 通过使用事件Kafka和PySpark集成来自8个数据源的1亿条原始记录,设计并实现了实时数据管道,处理半结构化和非结构化数据,并将处理后的数据存储到Teradata中.
- 通过在数据库上创建索引和更改业务逻辑,分析了ETL管道所花费的时间和改进的性能.
- 使用Azure数据工厂完成一个从AWS云到Azure的迁移项目, data migration, Databricks, Azure storage, Container, and Event Hub.
Software Engineer
Trinus
- 使用shell脚本将SSIS ETL代码迁移到Informatica PowerCenter中.
- 创建作业监控进程,每日查看所有作业状态, fixed the process, and delivered the data to the end user.
- Used Informatica Power Center for ETL extraction, transformation, 并将数据从异构源系统加载到目标数据库中. 使用Informatica web服务转换角色从web服务中提取数据.
Software Engineer
The Digital Group
- 在从遗留应用程序迁移项目的法律和金融领域工作. 从Oracle Pro*C和SQL加载器到数据仓库,使用Informatica ETL工具和Unix shell脚本.
- 使用Informatica ETL和Unix shell脚本将应用程序Oracle Pro*C代码转换为管道. 在staging层上创建自动数据质量配置文件规则,以清理数据并在仓库中捕获.
- 使用表创建报告,数据传递速度很快. 我还捕获了客户的历史信息,以便客户可以看到客户如何更改其信息.
Experience
eCommerce and Manufacturing Data Warehouse
Outcomes:
• Extracted data from different source systems, including ERP, relational, real-time events, and files, and loaded Datalake, a database staging layer.
•根据客户需求实现各种业务规则,并在报表层将数据交付给客户.
•创建端到端的数据流,直到报表层.
• Built Delta warehouse in Databricks.
Tools and databases:
Datalake, staging, ods, and EDW.
Data Migration to Public Cloud
Education
Bachelor's Degree in Information Technology
Mumbai University - Mumbai, India
High School Diploma in Science
Amravati University - Amaravati, India
Certifications
Hands On Essentials | Data Warehouse
Snowflake
Microsoft Certified | Azure Fundamentals
Microsoft
Skills
Libraries/APIs
PySpark, Pandas, REST APIs
Tools
Jira, SQL Management Studio, Informatica PowerCenter, Microsoft Excel, Terraform, Apache Airflow, Microsoft Power BI, Matillion ETL for Redshift, Informatica ETL, Control-M, Azure DevOps Services, Azure Logic Apps, Azure Automation, Microsoft Report Builder, Informatica Master Data Management (MDM), Kibana, Microsoft PowerPoint, Azure IoT Hub, Toad, Spark SQL, PyCharm, Microsoft Power Apps
Frameworks
ADF, Windows PowerShell, Flask
Paradigms
商业智能,数据库设计,Azure DevOps, ETL, DevOps
Languages
SQL, T-SQL (Transact-SQL), SQL DDL, Python, Snowflake, Python 3
Platforms
Azure, Microsoft, Databricks, Unix, Azure Synapse, Oracle, Amazon EC2, Azure Event Hubs, Azure Functions, Azure SQL Data Warehouse, Visual Studio 2017, Linux, Dedicated SQL Pool (formerly SQL DW), Amazon Web Services (AWS), Visual Studio Code (VS Code)
Storage
Azure Cloud Services, Microsoft SQL Server, SQL Server Integration Services (SSIS), SSAS Tabular, Teradata, Data Integration, SQL Server Reporting Services (SSRS), SQL Stored Procedures, Database Architecture, Database Migration, Redshift, Azure SQL Databases, Database Security, SQL Server 2017, MySQL, JSON, Data Lakes, SQL Server Analysis Services (SSAS), Databases, Elasticsearch
Other
Data Warehousing, Data Analytics, ETL Tools, Data Modeling, Azure Data Lake, Azure Data Factory, Data Architecture, Data Vaults, Data Warehouse Design, Azure Analysis Services, Analytics, Informatica, Data Analysis, Large Data Sets, Microsoft Azure Cloud Server, APIs, Data Governance, Unix Shell Scripting, ServiceNow, ARM, MSBI, Cloud, Cloud Architecture, Delta Lake, Azure Stream Analytics, Azure Data Lake Analytics, Data Engineering, SSRS Reports, HTTP REST, Workbench, Azure Administrator, SQL Server Administration, Cloud Infrastructure, Microsoft 365, erwin Data Modeler, Informatica Data Quality
How to Work with Toptal
在数小时内,而不是数周或数月,我们的网络将为您直接匹配全球行业专家.
Share your needs
Choose your talent
Start your risk-free talent trial
Top talent is in high demand.
Start hiring