Hi, I'm Min (Mia) Shi.
A
Welcome to my GitHub page! I am a dedicated Data Analyst, Data Scientist, and Data Engineer with a passion for learning, sharing knowledge, and making impactful contributions. With a diverse portfolio of internships and project experience, I am actively seeking a full-time role in Data Analysis, Data Science, or Data Engineering, starting in December 2024. I am excited to bring my expertise to your team and help drive data-driven innovation in your organization!
About
I am advancing my expertise in Data Science and Data Engineering through my education at UTD and my diverse intern and project experiences.
At Sunwater Institute, I serve as a Data Scientist, where I manage end-to-end data processes, from collection to infrastructure support. I’ve developed and optimized data pipelines and implemented ETL processes using AWS S3 and Glue, which increased data processing efficiency by 40%. I also automated speech-to-text and speaker identification with AWS Transcribe, achieving over 99% accuracy, and managed the underlying infrastructure, including database and system administration. My work ensures that our data systems are both scalable and user-friendly.
As a Data Scientist Student Consultant working on Onyx's project, I spearheaded the development of an AI-driven chatbot using advanced NLP techniques. This project significantly enhanced customer interaction and increased engagement by 15% through improved response efficiency. My work with NLP and machine learning, particularly utilizing the XGBoost model, advanced the company’s customer service capabilities.
During my four-year Data Analyst & Research Assistant work, I led data analysis for over 10 global health and policy projects using advanced statistical models. I managed data collection through various methods, including Qualtrics surveys and web scraping with R and Python. I developed more than 20 robust statistical models, integrating machine learning and NLP techniques to support correlation and causal inference in research. Additionally, I led a team of five junior assistants, ensuring effective collaboration and the timely completion and publication of our projects.
Eager to apply my data skills and analytical acumen, I am excited about the opportunity to contribute to your team and push the boundaries of what data can achieve in the tech industry.
- Programming Languages: Python, R, SQL, SAS, Stata, Java
- Databases: MySQL, PostgreSQL, SQL Server, MongoDB, Amazon RDS, Amazon S3
- Big Data: Hadoop, Sqoop, Hive, Impala, Pig, Spark
- Automation: Alteryx, Appian, Accelq, Uipath
- Visualization Tools: Tableau, Power BI, Jupyter Notebook, R Shiny
- Libraries: NumPy, pandas, Matplotlib, scikit-learn, SciPy, Statsmodels, nltk, PyTorch, Tensorflow
- Certificate: Graduate Certificate in Applied Machine Learning at UTD, Google Data Analytics Certificate, AWS Certified Cloud Practitioner Certificate, Alteryx Designer Core Certificate, Appian Certified Associate Developer, ACCELQ Automation Engineer Certification
- Languages: English, Chinese, Japanese
Experience
- Summary: Managed data collection, developed data pipeline, build analytics dashboards, and provide infrastructure support.
- Implemented web scraping solutions to extract data from websites, storing over 1 million records in databases.
- Created ETL process for ingesting data using AWS S3 and Glue, boosting data processing efficiency by 40%.
- Automated speech-to-text and speaker identification using AWS Transcribe, achieving over 99% accuracy.
- Developing and maintaining interactive Shiny dashboards, ensuring scalability and usability.
- Managing supporting infrastructure, including database and system administration.
- Tools: Python, Microsoft SQL Server, AWS S3, AWS Glue, AWS Transcribe, R Shiny
- Summary: Took responsibility for data analysis for 10+ global health/policy projects using advanced statistical models.
- Managed data collection in diverse methods including Qualtrics surveys and web scraping using R and Python.
- Developed 20+ robust statistical models (multi-variable and fixed-effect regression, difference-in-difference, time-series) combined ML models and NLP skills to support correlation and causal inference in research.
- Led a team of five junior assistants, ensuring collaboration and timely project completion and publication.
- Tools: Causal Analytics, A/B Testing, Data Collection, Data Analytics, Python, R, Stata
- Summary: Led the creation of an AI-driven chatbot, enhancing customer engagement through advanced NLP techniques.
- Employed NLP and MySQL for analyzing and querying an extensive database containing over 10 million entries.
- Achieved 25% improvement in response efficiency and provided 99% accurate predictions using XGBoost model.
- Contributed to a 15% rise in user engagement, increasing customer satisfaction and bolstering company’s image.
- Tools & Skills: Python, SQL, NLP, ML, UI Design, Leadership, Communication
- Summary: Served as a Data Analyst Intern responsible for data management, data visualization, and business analysis.
- Improved the efficiency of data extraction by 40% through data optimization in MySQL.
- Employed Microsoft Visio to visualize intricate network structures and aided in product comprehension.
- Produced Business Intelligence (BI) reports, offering insights based on user structures and competitor analysis.
- Tools: MySQL, Microsoft Visio, Microsoft Office
Projects
![Twitter Clone: High-throughput Social Media Backend](/assets/img/Twitter-X-logo.png)
A six-month solo project developing a social media platform’s backend using HBase, MySQL, and Redis with Django framework in Python.
- Tools: Python · Django · AWS · HBase · MySQL · Redis
- Maximizing query efficiency by storing objects with HBase, MySQL & Amazon S3 based on query complexity.
- Addressing N+1 slow query issues by implementing Redis caching and denormalization.
- Integrating Celery and RabbitMQ to establish asynchronous workers with varying priority levels.
- Implementing a push model for distributing news feeds to followers efficiently.
- Optimizing memory and resource allocation using recursive small batches of asynchronous tasks.
![US Top 4 Airlines Financial Performance Analytics Project](/assets/img/Airlines.jpeg)
Comprehensive Financial Performance Analytics of the Top Four US Airlines
- Tools: Data Mining · Business Model Analytics
- Analyzed financial data from a 20-year dataset of over 10,000 rows, covering net income, revenue, and expenses across the US airline industry. This deep dive provided insights into long-term financial trends and shifts.
- Conducted financial performance analytics for the top 4 airlines, identifying key turning points related to major events, alliances, and partnerships over the period.
- Assessed operational trends and competitive positioning of each airline, deriving specific business model recommendations based on a two-decade comparison with competitors.
![kaggle plant competition project](/assets/img/kaggle_plant_logo.png)
Leveraging Deep Learning CNNs for Disease Diagnosis in Apple Orchards
- Tools: Deep Learning · CNN · Transfer Learning
- Utilized transfer learning on CNNs with 5,590 images in 12 categories, enhancing disease identification accuracy.
- Conducted image transformation, including rotation, flipping, zooming, and noise injections to augment data.
- Fine-tuned ConvNext DL CNN models and achieve 86.8% accuracy, securing a Top 3 ranking in the competition.
![django web app](/assets/img/NLP.jpg)
Leveraging Web Scraping for Business Prediction via NLP & ML Approaches
- Tools: Web Scraping · Natural Language Processing · Machine Learning Models
- Created an automated web scraping tool to extract more than 7,000 WSJ news articles using specified keywords.
- Analyzed WSJ articles employing Count Vectorizer, Tfidf Vectorizer, and n-grams Count Vectorizer.
- Implemented Naïve Bayes and Random Forest models; achieved a notable ROC AUC value and an increase in S&P 500 stock index prediction accuracy by 12%.
- Demonstrated consistent accuracy across various vectorizers, suggesting the potential use of NLP in forecasting stock price changes based on WSJ news articles related to U.S. trade.
![](/assets/img/Trucks.jpg)
Geospatial Truck Fleet Big Data Analytics and Visualization
- Tools: Hadoop ecosystem · Tableau · R
- Used big data Hadoop ecosystem to process geospatial data ingestion, transformation, and database creation.
- Performed data exploration and visualization in Tableau by connecting to Hadoop ecosystem server.
- Modeled how factors affect the truck driver risk factor, drew a final report and proposed suggestions on how to lower the probability of large trucks accidents.
![](/assets/img/Conagra_logo.jpeg)
Extensive Analysis of Table Spreads Industry
- Tools: SAS · Tableau · Statistical Regression Analysis · Time Series Analysis
- Researched over 1.3 million records to identify key metrics contributing to the sales of top brands
- Evaluated strengths and weakness of Conagra Brands compared to competitors in each sub-category
- Built Machine Learning and Time Series models to predict future directions for Conagra Brands
![](/assets/img/Database.jpg)
Payroll Management System Database Design via MySQL
- Tools: MySQL
- Led a group of five in conducting business requirements analysis and designing a payroll management database with MySQL consisting of 13 tables.
- Increased efficiency in extract-transform-load and payroll database management by 100% via stored functions, procedures, and triggers.
![](/assets/img/Goldman-Sachs.jpeg)
Goldman Sachs Global Business Analytics and Prediction via Python and Alteryx
- Tools: Alteryx · Python · Business Analytics · Time Series Analysis
- Researched and generated datasets of US and worldwide inflation during pre-pandemic and post-pandemic periods via Python from raw datasets
- Conducted data cleaning and preprocessing, built time series ARIMA and ETS models to forecast trends in Consumer Price Index (CPI) and Producer Price Index (PPI) via Alteryx
- Presented key trends and findings on inflation and consumer prices and the further impact on Goldman Sachs, provided insights and recommendations on global operation strategies based on the analysis
![](/assets/img/COVID_USFirms.jpeg)
Analysis of the Effect of COVID-19 on US Trade and US Firms
- Tools: Deep Learning · Machine Learning · Statistical Regression Analysis · Communication
- Synthesized data and created fixed-effect regression models to identify correlations and causal mechanisms
- Developed and Implemented machine learning and deep learning models to conduct counterfactual analysis
- Presented findings at the 2023 Applied Data Science international conference
![](/assets/img/US-China_Trade.jpeg)
Modeling U.S.-China Trade War’s effect on US Multinational Corporations
- Tools: Python · R · SQL · Stata · Time Series Analysis
- Generated and managed a new database using PostgreSQL and performed data analysis in Python
- Built time series GARCH models in Stata to examine the effects of U.S.-China trade conflicts on US firms
- Presented the findings at the 2022 International Society for Data Science and Analytics Conference
![django web app](/assets/img/COVID19_Dashboard.png)
COVID-19 Worldwide Cases Synchronous Dashboard using Tableau
![](/assets/img/MNC.jpeg)
U.S. Multinational Corporation Trade Database with China
- Tools: SQL · Databases · MySQL
- The database aims to serve people interested in US-China trade war and its effect on US-China trade volumes, and the impact on US multinational corporations (MNCs) which depend on global value chains (GVC) heavily.
- The database provides mainly two types of data. The first type is the macroscopic data. Specifically, this database provides US-China monthly trade data by commodity, the volume and percentage of products under tariff data between US and China. It also contains the data about US annual trade with all countries and the basic development indicator information of these countries, including GDP, population, tariff rate in general, and tariff rate for manufactured products. The second type of data is the microcosmic data about MNCs, including S&P 500 company list with detailed information, such as stock symbol, location, sector, industry, etc., S&P 500 company stock price time-series data, fortune 500 company list and their annual revenues data, fortune 500 company stock price time-series data, top 20 companies list based on their level of sale in China, and top 20 companies list based on their share of sale in China.
- People could utilize this dataset to explore how U.S.-China trade relations change in the 21st century, the connection between U.S.-China trade and their tariff change, the differences in the tendencies of US trade with different companies, how U.S.-China trade relations affect U.S. multinational corporations (MNCs).
![](/assets/img/GNH.png)
What Factors Affect People’s National Happiness Score?
- Tools: R · R Markdown
- The World Happiness Report is the most professional annual report about countries’ happiness index and has attracted attention from policymakers from multiple areas. The values of happiness scores are based on respondent ratings of their own lives. And each report also includes six basic factors covering financial generation, social back, life anticipation, flexibility, nonattendance of debasement, and liberality.
- This paper explores the effect of other potential factors, including regime type, demographic factors, and COVID-19 severity. The statistical findings indicate that the more democratic one country is, and the larger the population size, its citizens tend to feel happier. Besides, the population density, population net change, and population density net change are negatively correlated to one country’s happiness score.
- Two interesting findings through data visualization are: Firstly, the relationship between democracy and one country’s happiness index follows a U shape rather than a positive linear line, different from the statistical results and no support to H1. Secondly, the population size and population density are negatively related to one country’s happiness index, supporting H2a and H2b. The results indicate that statistical regression results are not reliable in all cases, and data visualization is necessary to examine and interpret the statistical regression results more accurately.
Skills
Programming Languages
![](/assets/img/python-logo-1-300x300.jpg)
![](/assets/img/SQL-logo.png)
![](/assets/img/R_logo.png)
![](/assets/img/java.jpeg)
Tools
![](/assets/img/aws-web-services.png)
![](/assets/img/shiny.png)
![](/assets/img/SAS_logo.jpeg)
![](/assets/img/Stata_logo.png)
![](/assets/img/Tableau-logo.jpg)
![](/assets/img/Alteryx_logo.png)
![](/assets/img/Appian_logo.png)
![](/assets/img/ACCELQ_logo.png)
![](/assets/img/UiPath_logo.png)
![](/assets/img/Jupyter_Notebook.jpeg)
![](/assets/img/Excel-logo.png)
Database & ETL & Big Data
![](/assets/img/mysql-logo-1-300x300.jpg)
![](/assets/img/postgresql-logo.png)
![](/assets/img/SQL-Server.png)
![](/assets/img/Mango-DB-logo.png)
![](/assets/img/AmazonRDS-logo.png)
![](/assets/img/AWS-S3.png)
![](/assets/img/AWS-Glue.png)
![](/assets/img/hadoop-logo.png)
![](/assets/img/Sqoop-logo.png)
![](/assets/img/Hive-logo.png)
![](/assets/img/Impala-logo.jpeg)
![](/assets/img/Pig-logo.png)
![](/assets/img/Spark-logo.jpeg)
Libraries
![](/assets/img/numpy-logo-1-500x500.jpg)
![](/assets/img/pandas-logo.png)
![](/assets/img/sk-learn-logo-1-500x500.jpg)
![](/assets/img/SciPy-logo.png)
![](/assets/img/nltk-logo.png)
![](/assets/img/statsmodels-logo.png)
![](/assets/img/PyTorch.png)
![](/assets/img/Selenium-logo.png)
![](/assets/img/boto-3-logo.jpeg)
Certificates
![](/assets/img/UTD.png)
![](/assets/img/Google-data-analytics.png)
![](/assets/img/aws-certified-cloud-practitioner.png)
![](/assets/img/alteryx-designer-core-certification.png)
![](/assets/img/Appian_Certificate.png)
![](/assets/img/ACCELQ_Certificate.png)
Languages
![](/assets/img/English.png)
![](/assets/img/Chinese.png)
![](/assets/img/Japanese.png)
Education
The University of Texas at Dallas
Dallas, USA
Degree: Ph.D. Candidate in Political Science – Quantitative Statistical Modeling Focused
GPA: 3.95/4.0
Degree: Master of Science in Business Analytics (STEM) – Data Science & Data Engineer Track
GPA: 4.0/4.0
Degree: Master of Science in Social Data Analytics and Research
GPA: 3.95/4.0
Degree: Master of Art in Political Science
GPA: 3.95/4.0
Jinan, China
Degree: Master of Law in International Politics
GPA: 88.78/100
Degree: Bachlor of Arts in Japanese
GPA: 87.37/100