{"id":3444,"date":"2024-09-01T16:33:01","date_gmt":"2024-09-01T16:33:01","guid":{"rendered":"https:\/\/workhouse.sweetdishy.com\/?p=3444"},"modified":"2024-09-01T16:33:02","modified_gmt":"2024-09-01T16:33:02","slug":"backpropagation","status":"publish","type":"post","link":"https:\/\/workhouse.sweetdishy.com\/index.php\/2024\/09\/01\/backpropagation\/","title":{"rendered":"Backpropagation"},"content":{"rendered":"\n

One of the major drawbacks with artificial neural networks is the process of making adjustments to the weights in the model. Traditional approaches, like the use of the mutation algorithm, used random values that proved to be time consuming.<\/p>\n\n\n\n

Given this,\u00a0researchers\u00a0looked for alternatives, such as backpropagation. This technique had been around since the 1970s but got little interest as the performance was lacking. But David Rumelhart,\u00a0Geoffrey Hinton, and Ronald Williams realized that backpropagation still had potential, so long as it was refined. In 1986, they wrote a paper entitled \u201cLearning Representations by Back-propagating Errors,\u201d and it was a bombshell in the AI community.4<\/sup>\u00a0It clearly showed that backpropagation could be much faster but also allow for more powerful artificial neural\u00a0networks\u00a0.<\/p>\n\n\n\n

As should be no surprise, there is a lot of math involved in backpropagation. But when you boil things down, it\u2019s about adjusting the neural network when errors are found and then iterating the new values through the neural network again. Essentially, the process involves slight changes that continue to optimize the model.<\/p>\n\n\n\n

For example, let\u2019s say one of the inputs has an output of 0.6. This means that the error is 0.4 (1.0 minus 0.6), which is subpar. But we can then backpropogate the output, and perhaps the new output may get to 0.65. This training will go on until the value is much closer to 1.<\/p>\n\n\n\n

Figure\u00a04-3\u00a0illustrates this process. At first, there is a high level of errors because the weights are too large. But by making iterations, the errors will gradually fall. However, doing too much of this could mean an increase in errors. In other words, the goal of\u00a0backpropagation\u00a0is to find the midpoint.<\/p>\n\n\n\n

\"..\/images\/480660_1_En_4_Chapter\/480660_1_En_4_Fig3_HTML.jpg\"\/
Figure 4-3.<\/em><\/strong>The optimal value for a backpropagation function is at the bottom of the graph<\/figcaption><\/figure>\n\n\n\n

As a gauge of the success of backpropagation, there were a myriad of commercial applications that sprung up. One was called NETtalk, which was developed by Terrence Sejnowski and Charles Rosenberg in the mid-1980s. The machine was able to learn how to pronounce English text. NETtalk was so interesting that it was even demoed on the Today show<\/em>.<\/p>\n\n\n\n

There were also a variety of startups that were created that leveraged backpropagation, such as HNC Software. It built models that detected credit card fraud. Up until this point\u2014when HNC was founded in the late 1980s\u2014the process was done mostly by hand, which led to costly errors and low volumes of issuances. But by using deep learning approaches, credit card companies were able to save billions of dollars.<\/p>\n\n\n\n

In 2002, HNC was acquired by Fair, Isaac and valued at $810\u00a0million.5<\/sup><\/p>\n","protected":false},"excerpt":{"rendered":"

One of the major drawbacks with artificial neural networks is the process of making adjustments to the weights in the model. Traditional approaches, like the use of the mutation algorithm, used random values that proved to be time consuming. Given this,\u00a0researchers\u00a0looked for alternatives, such as backpropagation. This technique had been around since the 1970s but […]<\/p>\n","protected":false},"author":1,"featured_media":3327,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[442],"tags":[],"class_list":["post-3444","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-deep-learning"],"jetpack_featured_media_url":"https:\/\/workhouse.sweetdishy.com\/wp-content\/uploads\/2024\/08\/deep-learning-1.png","_links":{"self":[{"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/posts\/3444","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/comments?post=3444"}],"version-history":[{"count":1,"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/posts\/3444\/revisions"}],"predecessor-version":[{"id":3445,"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/posts\/3444\/revisions\/3445"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/media\/3327"}],"wp:attachment":[{"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/media?parent=3444"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/categories?post=3444"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/tags?post=3444"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}