{"id":3452,"date":"2024-09-01T16:42:59","date_gmt":"2024-09-01T16:42:59","guid":{"rendered":"https:\/\/workhouse.sweetdishy.com\/?p=3452"},"modified":"2024-09-01T16:42:59","modified_gmt":"2024-09-01T16:42:59","slug":"drawbacks-with-deep-learning","status":"publish","type":"post","link":"https:\/\/workhouse.sweetdishy.com\/index.php\/2024\/09\/01\/drawbacks-with-deep-learning\/","title":{"rendered":"Drawbacks with\u00a0Deep Learning"},"content":{"rendered":"\n<p id=\"Par168\">Given all the innovations and breakthroughs, it\u2019s reasonable that many people consider deep learning to be a silver bullet. It will mean we no longer have to drive a car. It may even mean that we\u2019ll cure cancer.<\/p>\n\n\n\n<p id=\"Par169\">How is it not possible to be excited and optimistic? This is natural and reasonable. But it is important to note that deep learning is still in a nascent stage and there are actually many nagging issues. It\u2019s a good idea to temper expectations.<\/p>\n\n\n\n<p>In 2018,\u00a0Gary Marcus\u00a0wrote a paper entitled \u201cDeep Learning: A Critical Appraisal,\u201d in which he clearly set forth the challenges.<sup>35<\/sup>\u00a0In his paper, he notes:<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p id=\"Par172\">Against a background of considerable\u00a0progress\u00a0in areas such as speech recognition, image recognition, and game playing, and considerable enthusiasm in the popular press, I present ten concerns for deep learning, and suggest that deep learning must be supplemented by other techniques if we are to reach Artificial General Intelligence.<sup>36<\/sup><\/p>\n<\/blockquote>\n\n\n\n<p id=\"Par174\">Marcus\u00a0definitely has the right pedigree to present his concerns, as he has both an academic and business background in AI. Before becoming a professor at the Department of Psychology at New York\u00a0University, he sold his startup, called Geometric Intelligence, to Uber. Marcus is also the author of several bestselling books like\u00a0<em>The Haphazard Construction of the Human Mind<\/em>.<sup>37<\/sup><\/p>\n\n\n\n<p>Here\u2019s a look at some of his worries about deep learning:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><em>Black Box<\/em>: A deep learning model could easily have millions of parameters that involve many hidden layers. Having a clear understanding of this is really beyond a person\u2019s capabilities. True, this may not necessarily be a problem with recognizing cats in a dataset. But it could definitely be an issue with models for medical diagnosis or determining the safety of an oil rig. In these situations, regulators will want to have a good understanding of the transparency of the models. Because of this, researchers are looking at creating systems to determine \u201cexplainability,\u201d which provides an understanding of deep learning models.<\/li>\n\n\n\n<li><em>Data<\/em>: The human brain has its flaws. But there are some functions that it does extremely well like the ability to learn by&nbsp;abstraction. For example, suppose Jan, who is five years old, goes to a restaurant with her family. Her mother points out an item on the plate and says it is a \u201ctaco.\u201d She does not have to explain it or provide any information about it. Instead, Jan\u2019s brain will instantly process this information and understand the overall pattern. In the future, when she sees another taco\u2014even if it has differences, such as with the dressing\u2014she will know what it is. For the most part, this is intuitive. But unfortunately, when it comes to deep learning, there is no taco learning by abstraction! The system has to process enormous amounts of information to recognize it. Of course, this is not a problem for companies like Facebook,&nbsp;Google, or even Uber. But many companies have much more limited datasets. The result is that deep learning may not be a good&nbsp;option.<\/li>\n\n\n\n<li><em>Hierarchical Structure<\/em>: This way of organizing does not exist in deep learning. Because of this, language understanding still has a long way to go (especially with long discussions).<\/li>\n\n\n\n<li><em>Open-Ended Inference<\/em>:&nbsp;Marcus&nbsp;notes that deep learning cannot understand the nuances between \u201cJohn promised Mary to leave\u201d and \u201cJohn promised to leave Mary.\u201d What\u2019s more, deep learning is far away from being able to, for&nbsp;instance, read Jane Austen\u2019s&nbsp;<em>Pride and Prejudice<\/em>&nbsp;and be able to divine Elizabeth Bennet\u2019s character motivations.<\/li>\n\n\n\n<li><em>Conceptual Thinking<\/em>: Deep learning cannot have an understanding of concepts like democracy, justice, or happiness. It also does not have imagination, thinking of new ideas or plans.<\/li>\n\n\n\n<li><em>Common Sense<\/em>: This is something deep learning does not do well. If anything, this means a model can be easily confused. For example, let\u2019s say you ask an AI system, \u201cIs it possible to make a computer with a sponge?\u201d For the most&nbsp;part, it will probably not know that this is a ridiculous question.<\/li>\n\n\n\n<li><em>Causation<\/em>: Deep learning is unable to determine this. It\u2019s all about finding correlations.<\/li>\n\n\n\n<li><em>Prior Knowledge<\/em>:&nbsp;CNNs&nbsp;can help with some prior information, but this is limited. Deep learning is still fairly self-contained, as it only solves one problem at a time. It cannot take in the data and create algorithms that span various domains. In addition, a model does not adapt. If there is change in the data, then a new model needs to be trained and tested. And finally, deep learning does not have prior understanding of what people know instinctively\u2014such as basic physics of the real world. This is something that has to be explicitly programmed into an AI system.<\/li>\n\n\n\n<li><em>Static<\/em>: Deep learning works best in environments that are fairly simple. This is why AI has been so effective with board&nbsp;games, which have a clear set of rules and boundaries. But the real world is chaotic and unpredictable. This means that deep learning may fall short with complex problems, even with self-driving cars.<\/li>\n\n\n\n<li><em>Resources<\/em>: A deep learning model often requires a tremendous amount of CPU power, such as with GPUs. This can get costly. Although, one option is to use a third-party cloud service.<\/li>\n<\/ul>\n\n\n\n<p>This is quite a lot? It\u2019s true. But the paper still has left out some drawbacks. Here are a couple other ones:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><em>Butterfly Effect<\/em>: Because of the complexity of the data, networks, and connections, a minute change can have a major impact in the results of the deep learning&nbsp;model. This could easily lead to conclusions that are wrong or misleading.<\/li>\n\n\n\n<li><em>Overfitting<\/em>&nbsp;: We explained this concept earlier in the chapter.<\/li>\n<\/ul>\n\n\n\n<p id=\"Par190\">As for\u00a0Marcus, his biggest fear is that AI could \u201cget trapped in a local minimum, dwelling too heavily in the wrong part of intellectual space, focusing too much on the detailed exploration of a particular class of accessible but limited models that are geared around capturing low-hanging fruit\u2014potentially neglecting riskier excursions that might ultimately lead to a more robust path.\u201d<\/p>\n\n\n\n<p id=\"Par191\">However, he is not a pessimist. He believes that researchers need to go beyond&nbsp;deep learning&nbsp;and find new techniques that can solve tough problems.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Given all the innovations and breakthroughs, it\u2019s reasonable that many people consider deep learning to be a silver bullet. It will mean we no longer have to drive a car. It may even mean that we\u2019ll cure cancer. How is it not possible to be excited and optimistic? This is natural and reasonable. But it [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":3327,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[442],"tags":[],"class_list":["post-3452","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-deep-learning"],"jetpack_featured_media_url":"https:\/\/workhouse.sweetdishy.com\/wp-content\/uploads\/2024\/08\/deep-learning-1.png","_links":{"self":[{"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/posts\/3452","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/comments?post=3452"}],"version-history":[{"count":1,"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/posts\/3452\/revisions"}],"predecessor-version":[{"id":3453,"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/posts\/3452\/revisions\/3453"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/media\/3327"}],"wp:attachment":[{"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/media?parent=3452"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/categories?post=3452"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/tags?post=3452"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}