Fireside Chat with
Dr. Harry Shum

March 11, 2020

« Blog

Recent achievements and advancements in artificial intelligence (AI) have outpaced what anyone would have thought imaginable even five to ten years ago. We are fast approaching human parity across many areas of AI — speech, vision, language, and knowledge. But many practitioners building AI technology and deploying AI products have not always thought through the societal implications such as fairness, ethics, and transparency.

“It’s critical to discuss how we can best address these societal challenges before the next AI innovation and development cycle. Up to this point, the answer has centered on principles – guidelines to help companies and countries navigate the complexities and implications of AI. But principles alone are no longer enough—industry, academia and government need to take actions now to move from principles to practices. I led the practices in Microsoft AI and Research from doing research in explainable and interpretable AI, to leveraging useful tools like datasheets and checklists commonly used in other industries, to forming an internal AI ethics committee providing guidelines for shipping AI products, to sharing and learning best practices with other companies through the Partnership in AI.”

Dr. Harry Shum

Former Executive Vice President, Microsoft

Dr. Harry Shum and Prof. Song-Chun Zhu at DMAI, Inc. on March 11, 2020

Harry Shum was Executive Vice President of Artificial Intelligence and Research at Microsoft until March 1, 2020. During his twenty-three year tenure, Dr. Shum was responsible for driving Microsoft’s overall AI strategy and forward-looking research and development efforts spanning infrastructure, services, apps, and agents. He supervised AI-focused product groups including Bing. Dr. Shum was one of the founding members of Microsoft Research China (later renamed Microsoft Research Asia). There he began as a researcher, and through his industry and insights, moved on to become Managing Director.

Dr. Shum received his Ph.D. in Robotics from the School of Computer Science at Carnegie Mellon University in 1996. He is an IEEE Fellow and an ACM Fellow for his contributions to computer vision and computer graphics. In 2017, Dr. Shum was elected to the National Academy of Engineering of the United States.

Once in a great while, the stars align, and they did so today, offering us privileged glimpses into the past and into the future of AI. Today we welcomed Dr. Harry Shum to the Los Angeles Headquarters of DMAI, Inc. where Professor Song-Chun Zhu, the founder of the DM Group, gave him a tour of the office.

But this was no ordinary office tour.

Having known each other for a quarter of a century, Dr. Shum and Professor Zhu, both heralded leaders in the development of a unified theory of artificial intelligence, have each watched and charted, with admiration, the career path of the other. Having watched each other from afar for so long, Dr. Shum and Professor Zhu joined forces today in their vision to develop a unified artificial intelligence that lifts humanity.

Following a day full of riveting conversations and product demonstrations, Dr. Shum set aside a bit of time to speak with us about the nature of startups and the future of AI, both of which he envisions as uniquely human endeavors.


Dr. Shum, welcome to DMAI. How do you think a startup like us can make a difference in AI innovation compared to large corporations like Microsoft, Google, and IBM?

Everyone has to start somewhere. Even if you consider the humble beginnings of Microsoft, you see Bill Gates in his half of a dorm room, and Paul Allen in the other half, and very similarly, you see the two Stanford kids that started Google. We all start from startups, and I think a startup company like DMAI has a bright future.

Dr. Shum, Prof. Zhu and you have had distinct career paths in AI, yet you have both collaborated for many years. What is the bond between you?

I came to know Professor Zhu when we were both Ph.D. students. I still remember the first time I saw him, at ICCV 1995. I saw this young energetic graduate student from this lowly university called “Harvard” [smiles] doing incredible work. All of us CMU [Carnegie Mellon University] students looked at Professor Zhu’s work and said, “Oh my God! This guy is doing something amazing!” So, ever since I’ve come to know Professor Zhu’s work, I have always admired his drive and his passion for getting to the core of what AI really is. We have followed each other’s careers, and now it’s been twenty-five years — a really amazing journey for both of us.

Here is a photograph of you with Prof. Zhu in 2006. Prof. Zhu created a non-profit institute in his hometown Ezhou, Hubei Province. The mission was to collect annotated image data, which marks the beginning of the data-driven paradigm and statistical learning in AI. What do you think are the biggest influences of AI on humanity since then? And what will be in the future?

Notes: Dr. Harry Shum and Prof. Song-Chun Zhu supervised students at the Lotus Hill Institute in Hubei, China in 2006.

This is a really amazing photograph. Thank you for sharing this. Clearly, Song-Chun and I look pretty young. One of the things very unique about the non-profit that Song-Chun created, and to which I contributed, is that, at that time, non-profits in computer vision were very rare. Song-Chun did pioneering work, and doing this work in China, in his hometown, was extraordinary. It was very difficult to build a non-profit to attract top talent in terms of researchers and students to work there, but Song-Chun did. That, in itself, is amazing. Song-Chun had the vision to dive into data-driven computer vision and data-driven AI at the early stage. He was really spot-on if you consider the future impact that the Ezhou Lotus Hill Institute has had, especially if you look at the success of image structures, which gives us a hint about the right direction of data-driven AI. It’s a really amazing accomplishment.

Dr. Shum, Prof. Zhu and you are both “integrators” of AI. What do you think are the necessary next steps to make integration happen?

One thing I’ve always liked about working with Professor Zhu in the areas of computer vision and AI is that we have a common understanding and belief that the right path to solve AI, to solve AGI is through integration. It’s not that one single problem that you solve in AI unlocks all its intelligence. That combination of elements, from perception to cognition to reasoning to causality, really have to come together. And something else I find unique about Professor Zhu is that he’s willing to get his hands dirty by bringing all those elements together; it’s natural for people like me in the industry who have to produce products. Therefore, you have to really integrate all those kinds of things. I think now the time has come to join forces, to look at this very exciting and very challenging area of artificial intelligence. But we also have to be realistic.

DMAI’s mission is to lift humanity. Dr. Shum, what do you think are the biggest opportunities and challenges in AI for our human race?

I like the mission statement of DMAI. It’s very similar to what we have at Microsoft. I believe in leveraging what we study — AI research — to help humanity, even today, as I visit DMAI and look at the exciting products you have here, at what you’re doing here: thinking about impact upon kids, especially kids with autism, what we can do, how we can apply AI to challenging situations. You don’t have to limit the technology you develop here at DMAI only to benefit young kids. You can imagine technology for the elderly, for anyone. Why do we grow more anxious everyday? Why are we so nervous? Just read the news, and you can feel restless. These matters, in my opinion, AI can help by lifting humanity, by lifting the spirit of humanity. It [DMAI’s mission statement] is such a noble cause.

Thank you Dr. Shum for visiting us today. What are your plans in the near future? Anything you can share with us?

Song-Chun and I studied in China, grew up in China, and we were very fortunate to come to the United States to learn and to study and to work here. What’s really exciting to me is that there’s really no boundary for AI. It’s not limited to a company, nor even to a country. We can all work together, do the right thing to develop AI technology jointly to serve humanity and to lift the spirit of humanity.


Dr. Delano Copprue
Editor-in-Chief
DMAI, Inc.

Milestones