{"id":3409,"date":"2024-09-01T13:44:47","date_gmt":"2024-09-01T13:44:47","guid":{"rendered":"https:\/\/workhouse.sweetdishy.com\/?p=3409"},"modified":"2024-09-01T13:44:48","modified_gmt":"2024-09-01T13:44:48","slug":"correlation","status":"publish","type":"post","link":"https:\/\/workhouse.sweetdishy.com\/index.php\/2024\/09\/01\/correlation\/","title":{"rendered":"Correlation"},"content":{"rendered":"\n<p id=\"Par55\">A&nbsp;machine learning&nbsp;algorithm often involves some type of correlation among the data. A quantitative way to describe this is to use the&nbsp;Pearson correlation, which shows the strength of the relationship between two variables that range from 1 to \u20131 (this is the coefficient).<\/p>\n\n\n\n<p>Here\u2019s how it works:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><em>Greater than 0<\/em>: This is where an increase in one variable leads to the increase in another. For example: Suppose that there is a 0.9 correlation between income and spending. If income increases by $1,000, then spending will be up by $900 ($1,000 X 0.9).<\/li>\n\n\n\n<li><em>0<\/em>: There is no correlation between the two variables.<\/li>\n\n\n\n<li><em>Less than 0<\/em>: Any increase in the variable means a decrease in another and vice versa. This describes an inverse relationship.<\/li>\n<\/ul>\n\n\n\n<p id=\"Par60\">Then what is a strong correlation? As a general rule of thumb, it\u2019s if the coefficient is +0.7 or so. And if it is under 0.3, then the correlation is tenuous.<\/p>\n\n\n\n<p id=\"Par61\">All this harkens the old saying of \u201cCorrelation is not necessarily causation.\u201d Yet when it comes to&nbsp;machine learning, this concept can easily be ignored and lead to misleading results.<\/p>\n\n\n\n<p>For example, there are many correlations that are just random. In fact, some can be downright comical. Check out the following from Tylervigen.com:<sup><a href=\"https:\/\/learning.oreilly.com\/library\/view\/artificial-intelligence-basics\/9781484250280\/html\/480660_1_En_3_Chapter.xhtml#Fn8\">8<\/a><\/sup><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>The divorce rate in Maine has a 99.26% correlation with per capita consumption of margarine.<\/li>\n\n\n\n<li>The age of Miss America has an 87.01% correlation with the murders by steam, hot vapors, and hot tropics.<\/li>\n\n\n\n<li>The US crude oil imports from Norway have a 95.4% correlation with drivers killed in collision with a railway train.<\/li>\n<\/ul>\n\n\n\n<p id=\"Par67\">There is a name for this: patternicity. This is the tendency to find patterns in meaningless noise.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>A&nbsp;machine learning&nbsp;algorithm often involves some type of correlation among the data. A quantitative way to describe this is to use the&nbsp;Pearson correlation, which shows the strength of the relationship between two variables that range from 1 to \u20131 (this is the coefficient). Here\u2019s how it works: Then what is a strong correlation? As a general [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":3410,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[441],"tags":[],"class_list":["post-3409","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-3-machine-learning"],"jetpack_featured_media_url":"https:\/\/workhouse.sweetdishy.com\/wp-content\/uploads\/2024\/09\/correlation.png","_links":{"self":[{"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/posts\/3409","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/comments?post=3409"}],"version-history":[{"count":1,"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/posts\/3409\/revisions"}],"predecessor-version":[{"id":3411,"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/posts\/3409\/revisions\/3411"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/media\/3410"}],"wp:attachment":[{"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/media?parent=3409"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/categories?post=3409"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/tags?post=3409"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}