<commit_before>---
layout:     post
title:      Data Science Journey At Microsoft
date:       2021-10-31 22:30
summary:    A recap of the 5-year data scientist journey at Microsoft
categories: technology, data science, machine learning, career development
---

<blockquote>
  <p>Always remember that when it comes to markets, past performance is not a good predictor of future returns—looking in the rear-view mirror is a bad way to drive. Machine learning, on the other hand, is applicable to datasets where the past is a good predictor of the future.</p>
  <footer><cite title="<NAME>"><NAME></cite></footer>
</blockquote>

![Redmond - 2017](https://github.com/yueguoguo/yueguoguo.github.io/raw/master/images/redmond_hq.JPG)
<font size="1">This is the first time I went to Microsoft Redmond campus. It was in the spring of 2017, which was my second year at Microsoft. I was astonished by the view and working environment of Redmond campus. What I will always recommend is <a href="https://dining.azurewebsites.net/34/">In.gredients</a> where you can have cheap but Michelin-quality food.</font>

## How the story began

**November 12th, 2021** marked my last day with Microsoft. I joined Microsoft as
a junior data scientist in 2016. Microsoft is my second company after I
graduated with the Ph.D. degree from Nanyang Technological University,
Singapore. My first company was Broadcom where I worked as a Product Engineer
for [wafer failure pattern
detection](https://ieeexplore.ieee.org/abstract/document/6932449). My title at
Broadcom was not "Data Scientist" though my job scope was completely about data
analytics and machine learning. That was when I started to learn about
industrial application of machine learning. Unlike many data scientists, I am
from an computer engineering background. In the last year of my Ph.D. and the
first several years of my industry career, I tried hard to develop the
statistical and analytical skills by reviewing my university courses, reading
books about data science and machine learning, attending conferences or meetups
to learn from experts, etc. 

When I just joined Microsoft as a Data Scientist, I think modelling is the
absolute core component of my job. After the first two years of various
engagements in the customer projects, I gradually realized that most data
scientists, including myself, in the industry do the "cool" work that prototyped
a nicely fitted model with unrealistically high-quality data without even
considering transforming the prototype into a production system. I suspect the
reasons behind this phenomenon are two fold: **users who "buy" the model do not
want to invest too much before they can see some appealing results; the
engineering platforms for applying data science into production is not powerful
nor standardized**. The observation forced me to rethink about the role of a
data scientist - I recognized the importance of combining software and computer
engineering with the machine learning to produce cool and, more importantly,
useable solutions, for users. 

At Microsoft, I was lucky to work with a team of talented people. I found a
great amount of interest similarities with my colleagues and that lead to
efficient ideation and execution. With their help, I participated in and
contributed to many impactful projects that created commercial value for
Microsoft customers. Below it is a selected list of such projects which
are publicly shareable (**we started open-source software development even
before Microsoft acquired GitHub!**).
<commit_msg>revise<commit_after>---
layout:     post
title:      My Learning about Machine Learning 
date:       2021-10-31 22:30
summary:    A recap of the 5-year data scientist journey at Microsoft
categories: technology, data science, machine learning, career development
---

<blockquote>
  <p>Always remember that when it comes to markets, past performance is not a good predictor of future returns—looking in the rear-view mirror is a bad way to drive. Machine learning, on the other hand, is applicable to datasets where the past is a good predictor of the future.</p>
  <footer><cite title="<NAME>"><NAME></cite></footer>
</blockquote>

![Redmond - 2017](https://github.com/yueguoguo/yueguoguo.github.io/raw/master/images/redmond_hq.JPG)
<font size="1">This is the first time I went to Microsoft Redmond campus. It was in the spring of 2017, which was my second year at Microsoft. I was astonished by the view and working environment of Redmond campus. What I will always recommend is <a href="https://dining.azurewebsites.net/34/">In.gredients</a> where you can have cheap but Michelin-quality food.</font>

## How the story began

**November 12th, 2021** marked my last day with Microsoft. I joined Microsoft as
a junior Data and Applied Scientist in 2016. Microsoft is my second company
after I graduated with the Ph.D. degree from Nanyang Technological University,
Singapore. My first company was Broadcom where I worked as a Product Engineer
for [wafer failure pattern
detection](https://ieeexplore.ieee.org/abstract/document/6932449). My job scope
at Broadcome was about data analytics and machine learning. That was when I
started to learn about industrial application of machine learning. Unlike many
data scientists, I am from an computer engineering background. In the last year
of my Ph.D. and the first several years of my industry career, I tried hard to
develop the statistical and analytical theories by reviewing my university
courses, reading books about data science and machine learning, attending
conferences or meetups to learn from experts, etc. 

When I just joined Microsoft as a Data Scientist, I think modelling is the
absolute core component of my job. After the first two years of various
engagements in the customer projects, I gradually realized that most data
scientists, including myself, in the industry do the "cool" work that prototyped
a nicely fitted model with unrealistically high-quality data without even
considering transforming the prototype into a production system. I suspect the
reasons behind this phenomenon are two fold: **users who "buy" the model do not
want to invest too much before they can see some appealing results; the
engineering platforms for applying data science into production is not powerful
nor standardized**. The observation forced me to rethink about the role of a
data scientist - I recognized the importance of combining software and computer
engineering with the machine learning to produce cool and, more importantly,
useable solutions, for users. 

At Microsoft, I was lucky to work with a team of talented people. I found a
great amount of interest similarities with my colleagues and that lead to
efficient ideation and execution. With their help, I participated in and
contributed to many impactful projects that created commercial value for
Microsoft customers. Below it is a selected list of such projects which
are publicly shareable (**we started open-source software development even
before Microsoft acquired GitHub!**).

{'fraction_non_alphanumeric': 0.050837830161885825, 'fraction_numerical': 0.011644419199091167, 'mean_word_length': 4.726016260162601, 'pattern_counts': {'":': 0, '<': 31, '<?xml version=': 0, '>': 31, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3819862', 'n_tokens_mistral': 1755, 'n_tokens_neox': 1622, 'n_words': 1014}