{"id":227,"date":"2019-09-10T17:43:02","date_gmt":"2019-09-10T17:43:02","guid":{"rendered":"http:\/\/104.42.120.246.xip.io\/calculus-tutorials\/?page_id=227"},"modified":"2020-06-18T16:55:25","modified_gmt":"2020-06-18T16:55:25","slug":"gram-schmidt-method","status":"publish","type":"page","link":"https:\/\/math.hmc.edu\/calculus\/hmc-mathematics-calculus-online-tutorials\/linear-algebra\/gram-schmidt-method\/","title":{"rendered":"Gram-Schmidt Method"},"content":{"rendered":"\n<script type=\"text\/x-mathjax-config\">\n  MathJax.Hub.Config({ tex2jax: { inlineMath: [['$','$'], [\"\\(\",\"\\)\"]] } });\n<\/script>\n\n\n\n<script type=\"text\/javascript\" src=\"http:\/\/cdn.mathjax.org\/mathjax\/latest\/MathJax.js?config=TeX-AMS_HTML\">\n<\/script>\n\n\n\n<meta http-equiv=\"X-UA-Compatible\" content=\"IE=EmulateIE7\">\n\n\n\n<title>The Gram-Schmidt Algorithm &#8211; HMC Calculus Tutorial<\/title>\n\n\n\n<p>In any <strong>inner product<\/strong> space, we can choose the basis in which to work.  <\/p>\n\n\n<style>.kt-accordion-id_019a0c-fe .kt-accordion-inner-wrap{column-gap:var(--global-kb-gap-md, 2rem);row-gap:0px;}.kt-accordion-id_019a0c-fe .kt-accordion-panel-inner{border-top-width:0px;border-right-width:0px;border-bottom-width:0px;border-left-width:0px;padding-top:var(--global-kb-spacing-sm, 1.5rem);padding-right:var(--global-kb-spacing-sm, 1.5rem);padding-bottom:var(--global-kb-spacing-sm, 1.5rem);padding-left:var(--global-kb-spacing-sm, 1.5rem);}.kt-accordion-id_019a0c-fe > .kt-accordion-inner-wrap > .wp-block-kadence-pane > .kt-accordion-header-wrap > .kt-blocks-accordion-header{border-top-color:#555555;border-right-color:#555555;border-bottom-color:#555555;border-left-color:#555555;border-top-width:0px;border-right-width:0px;border-bottom-width:0px;border-left-width:0px;border-top-left-radius:0px;border-top-right-radius:0px;border-bottom-right-radius:0px;border-bottom-left-radius:0px;background:#f2f2f2;font-size:18px;line-height:24px;color:#555555;padding-top:10px;padding-right:14px;padding-bottom:10px;padding-left:14px;}.kt-accordion-id_019a0c-fe:not( .kt-accodion-icon-style-basiccircle ):not( .kt-accodion-icon-style-xclosecircle ):not( .kt-accodion-icon-style-arrowcircle )  > .kt-accordion-inner-wrap > .wp-block-kadence-pane > .kt-accordion-header-wrap .kt-blocks-accordion-icon-trigger:after, .kt-accordion-id_019a0c-fe:not( .kt-accodion-icon-style-basiccircle ):not( .kt-accodion-icon-style-xclosecircle ):not( .kt-accodion-icon-style-arrowcircle )  > .kt-accordion-inner-wrap > .wp-block-kadence-pane > .kt-accordion-header-wrap .kt-blocks-accordion-icon-trigger:before{background:#555555;}.kt-accordion-id_019a0c-fe:not( .kt-accodion-icon-style-basic ):not( .kt-accodion-icon-style-xclose ):not( .kt-accodion-icon-style-arrow ) .kt-blocks-accordion-icon-trigger{background:#555555;}.kt-accordion-id_019a0c-fe:not( .kt-accodion-icon-style-basic ):not( .kt-accodion-icon-style-xclose ):not( .kt-accodion-icon-style-arrow ) .kt-blocks-accordion-icon-trigger:after, .kt-accordion-id_019a0c-fe:not( .kt-accodion-icon-style-basic ):not( .kt-accodion-icon-style-xclose ):not( .kt-accodion-icon-style-arrow ) .kt-blocks-accordion-icon-trigger:before{background:#f2f2f2;}.kt-accordion-id_019a0c-fe > .kt-accordion-inner-wrap > .wp-block-kadence-pane > .kt-accordion-header-wrap > .kt-blocks-accordion-header:hover, \n\t\t\t\tbody:not(.hide-focus-outline) .kt-accordion-id_019a0c-fe .kt-blocks-accordion-header:focus-visible{color:#444444;background:#eeeeee;border-top-color:#eeeeee;border-right-color:#eeeeee;border-bottom-color:#eeeeee;border-left-color:#eeeeee;}.kt-accordion-id_019a0c-fe:not( .kt-accodion-icon-style-basiccircle ):not( .kt-accodion-icon-style-xclosecircle ):not( .kt-accodion-icon-style-arrowcircle ) .kt-accordion-header-wrap .kt-blocks-accordion-header:hover .kt-blocks-accordion-icon-trigger:after, .kt-accordion-id_019a0c-fe:not( .kt-accodion-icon-style-basiccircle ):not( .kt-accodion-icon-style-xclosecircle ):not( .kt-accodion-icon-style-arrowcircle ) .kt-accordion-header-wrap .kt-blocks-accordion-header:hover .kt-blocks-accordion-icon-trigger:before, body:not(.hide-focus-outline) .kt-accordion-id_019a0c-fe:not( .kt-accodion-icon-style-basiccircle ):not( .kt-accodion-icon-style-xclosecircle ):not( .kt-accodion-icon-style-arrowcircle ) .kt-blocks-accordion--visible .kt-blocks-accordion-icon-trigger:after, body:not(.hide-focus-outline) .kt-accordion-id_019a0c-fe:not( .kt-accodion-icon-style-basiccircle ):not( .kt-accodion-icon-style-xclosecircle ):not( .kt-accodion-icon-style-arrowcircle ) .kt-blocks-accordion-header:focus-visible .kt-blocks-accordion-icon-trigger:before{background:#444444;}.kt-accordion-id_019a0c-fe:not( .kt-accodion-icon-style-basic ):not( .kt-accodion-icon-style-xclose ):not( .kt-accodion-icon-style-arrow ) .kt-accordion-header-wrap .kt-blocks-accordion-header:hover .kt-blocks-accordion-icon-trigger, body:not(.hide-focus-outline) .kt-accordion-id_019a0c-fe:not( .kt-accodion-icon-style-basic ):not( .kt-accodion-icon-style-xclose ):not( .kt-accodion-icon-style-arrow ) .kt-accordion-header-wrap .kt-blocks-accordion-header:focus-visible .kt-blocks-accordion-icon-trigger{background:#444444;}.kt-accordion-id_019a0c-fe:not( .kt-accodion-icon-style-basic ):not( .kt-accodion-icon-style-xclose ):not( .kt-accodion-icon-style-arrow ) .kt-accordion-header-wrap .kt-blocks-accordion-header:hover .kt-blocks-accordion-icon-trigger:after, .kt-accordion-id_019a0c-fe:not( .kt-accodion-icon-style-basic ):not( .kt-accodion-icon-style-xclose ):not( .kt-accodion-icon-style-arrow ) .kt-accordion-header-wrap .kt-blocks-accordion-header:hover .kt-blocks-accordion-icon-trigger:before, body:not(.hide-focus-outline) .kt-accordion-id_019a0c-fe:not( .kt-accodion-icon-style-basic ):not( .kt-accodion-icon-style-xclose ):not( .kt-accodion-icon-style-arrow ) .kt-accordion-header-wrap .kt-blocks-accordion-header:focus-visible .kt-blocks-accordion-icon-trigger:after, body:not(.hide-focus-outline) .kt-accordion-id_019a0c-fe:not( .kt-accodion-icon-style-basic ):not( .kt-accodion-icon-style-xclose ):not( .kt-accodion-icon-style-arrow ) .kt-accordion-header-wrap .kt-blocks-accordion-header:focus-visible .kt-blocks-accordion-icon-trigger:before{background:#eeeeee;}.kt-accordion-id_019a0c-fe .kt-accordion-header-wrap .kt-blocks-accordion-header:focus-visible,\n\t\t\t\t.kt-accordion-id_019a0c-fe > .kt-accordion-inner-wrap > .wp-block-kadence-pane > .kt-accordion-header-wrap > .kt-blocks-accordion-header.kt-accordion-panel-active{color:#ffffff;background:#444444;border-top-color:#444444;border-right-color:#444444;border-bottom-color:#444444;border-left-color:#444444;}.kt-accordion-id_019a0c-fe:not( .kt-accodion-icon-style-basiccircle ):not( .kt-accodion-icon-style-xclosecircle ):not( .kt-accodion-icon-style-arrowcircle )  > .kt-accordion-inner-wrap > .wp-block-kadence-pane > .kt-accordion-header-wrap > .kt-blocks-accordion-header.kt-accordion-panel-active .kt-blocks-accordion-icon-trigger:after, .kt-accordion-id_019a0c-fe:not( .kt-accodion-icon-style-basiccircle ):not( .kt-accodion-icon-style-xclosecircle ):not( .kt-accodion-icon-style-arrowcircle )  > .kt-accordion-inner-wrap > .wp-block-kadence-pane > .kt-accordion-header-wrap > .kt-blocks-accordion-header.kt-accordion-panel-active .kt-blocks-accordion-icon-trigger:before{background:#ffffff;}.kt-accordion-id_019a0c-fe:not( .kt-accodion-icon-style-basic ):not( .kt-accodion-icon-style-xclose ):not( .kt-accodion-icon-style-arrow ) .kt-blocks-accordion-header.kt-accordion-panel-active .kt-blocks-accordion-icon-trigger{background:#ffffff;}.kt-accordion-id_019a0c-fe:not( .kt-accodion-icon-style-basic ):not( .kt-accodion-icon-style-xclose ):not( .kt-accodion-icon-style-arrow ) .kt-blocks-accordion-header.kt-accordion-panel-active .kt-blocks-accordion-icon-trigger:after, .kt-accordion-id_019a0c-fe:not( .kt-accodion-icon-style-basic ):not( .kt-accodion-icon-style-xclose ):not( .kt-accodion-icon-style-arrow ) .kt-blocks-accordion-header.kt-accordion-panel-active .kt-blocks-accordion-icon-trigger:before{background:#444444;}@media all and (max-width: 767px){.kt-accordion-id_019a0c-fe .kt-accordion-inner-wrap{display:block;}.kt-accordion-id_019a0c-fe .kt-accordion-inner-wrap .kt-accordion-pane:not(:first-child){margin-top:0px;}}<\/style>\n<div class=\"wp-block-kadence-accordion alignnone\"><div class=\"kt-accordion-wrap kt-accordion-wrap kt-accordion-id_019a0c-fe kt-accordion-has-2-panes kt-active-pane-0 kt-accordion-block kt-pane-header-alignment-left kt-accodion-icon-style-basic kt-accodion-icon-side-right\" style=\"max-width:none\"><div class=\"kt-accordion-inner-wrap\" data-allow-multiple-open=\"true\" data-start-open=\"none\">\n<div class=\"wp-block-kadence-pane kt-accordion-pane kt-accordion-pane-3 kt-pane_42f756-23\"><div class=\"kt-accordion-header-wrap\"><button class=\"kt-blocks-accordion-header kt-acccordion-button-label-show\"><div class=\"kt-blocks-accordion-title-wrap\"><span class=\"kt-blocks-accordion-title\">Definition of Inner Product Space<\/span><\/div><div class=\"kt-blocks-accordion-icon-trigger\"><\/div><\/button><\/div><div class=\"kt-accordion-panel kt-accordion-panel-hidden\"><div class=\"kt-accordion-panel-inner\">\n<p>\n<!------------------------>\n\nAn <b>inner product space<\/b> is a real vector space $V$ with an inner product. Recall that an inner\nproduct $&lt; \\cdot, \\cdot &gt;$ is a function that, for each pair of vectors ${\\bf u}, {\\bf v} \\in V$, assigns a real number\nin such a way that\n<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li> $&lt; {\\bf u}, {\\bf v} &gt;\\ =\\ &lt; {\\bf v}, {\\bf u} &gt;$\n\t<\/li><li> $&lt; {\\bf u} + {\\bf v}, {\\bf w} &gt;\\ =\\ &lt; {\\bf u}, {\\bf w} &gt; + &lt; {\\bf v}, {\\bf w} &gt; {\\small\\textrm{ for all }} {\\bf w} \\in V.$\n\t<\/li><li> $&lt; k{\\bf u}, {\\bf v} &gt;\\ =\\ k &lt; {\\bf u}, {\\bf v} &gt;$\n\t<\/li><li> $&lt; {\\bf v}, {\\bf v} &gt;\\ \\geq 0, {\\small\\textrm{ where }} &lt; {\\bf v}, {\\bf v} &gt;\\ = 0 {\\small\\textrm{ if and only if }} {\\bf v} = 0.$\n<ol>\n\n<!------------------------>\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n<\/ol><\/li><\/ol>\n<\/div><\/div><\/div>\n<\/div><\/div><\/div>\n\n\n\n<p><br>It often greatly simplifies calculations to work in an orthogonal basis.  For one thing, if $ S = \\{ {\\bf v}_1, {\\bf v}_2, \\dots , {\\bf v}_n \\} $ is an <strong>orthogonal <\/strong>basis for an inner product space $V$, which means all pairs of distinct vectors in S are orthogonal: $$ &lt;  {\\bf v}_i, {\\bf v}_j  &gt;\\ = 0 {\\small\\textrm{ for all }}   {\\bf v}_i, {\\bf v}_j   \\in S. $$ <br>then it is a simple matter to express any vector ${\\bf w} \\in V$ as a linear combination of the vectors in $S$:<\/p>\n\n\n\n<p>\n\t\t$$\n\t\tw=\\frac{\\langle {\\bf w},{\\bf v}_1\\rangle}{\\| {\\bf v}_1 \\|^{2}}{\\bf v}_1 +\n\t\t\\frac{\\langle {\\bf w},{\\bf v}_2\\rangle}{\\| {\\bf v}_2 \\|^{2}}{\\bf v}_2 +\n\t\t\\cdots +\n\t\t\\frac{\\langle {\\bf w},{\\bf v}_n\\rangle}{\\| {\\bf v}_n \\|^{2}}{\\bf v}_n.\n\t\t$$\n\t\tGiven an arbitrary basis $\\{{\\bf u}_1, {\\bf u}_2, \\ldots, {\\bf u}_n\\}$ for an\n\t\t$n$-dimensional inner product space $V$, the <b>Gram-Schmidt\n\t\talgorithm<\/b> constructs an orthogonal basis $\\{{\\bf v}_1, {\\bf v}_2,\n\t\t\\ldots, {\\bf v}_n\\}$ for $V$:\n<\/p>\n\n\n\n<p>\n\t\t\tThat is, ${\\bf w}$ has coordinates\n\t\t\t$$\n\t\t\t\\left[\\begin{array}{c}\n\t\t\t\t\\frac{\\langle {\\bf w},{\\bf v}_1\\rangle}{\\| {\\bf v}_1 \\|^{2}}{\\bf v}_1\\\\\n\t\t\t\t\\frac{\\langle {\\bf w},{\\bf v}_2\\rangle}{\\| {\\bf v}_2 \\|^{2}}{\\bf v}_2\\\\\n\t\t\t\t\\vdots\\\\\n\t\t\t\t\\frac{\\langle {\\bf w},{\\bf v}_n\\rangle}{\\| {\\bf v}_n \\|^{2}}{\\bf v}_n\n\t\t\t\\end{array} \\right]\n\t\t\t$$\n\t\t\trelative to the basis $S$.\n<\/p>\n\n\n\n<p>\n<u>Step 1<\/u> Let ${\\bf v}_1 = {\\bf u}_1$.\n<\/p>\n\n\n\n<p> <u>Step 2<\/u> Let ${\\bf v}_2={\\bf u}_2 &#8211; \\mbox{proj}_{W_{1}}{\\bf u}_2 = {\\bf u}_2 &#8211; \\frac{\\langle {\\bf u}_2,{\\bf v}_1\\rangle}{\\| {\\bf v}_1 \\|^{2}}{\\bf v}_1$ where $W_1$ is the space spanned by ${\\bf v}_1$, and $\\mbox{proj}_{W_{1}}{\\bf u}_2$ is the <b>orthogonal projection<\/b> of ${\\bf u}_2$ on $W_1$.<\/p>\n\n\n<style>.kt-accordion-id_cc7480-e4 .kt-accordion-inner-wrap{column-gap:var(--global-kb-gap-md, 2rem);row-gap:0px;}.kt-accordion-id_cc7480-e4 .kt-accordion-panel-inner{border-top-width:0px;border-right-width:0px;border-bottom-width:0px;border-left-width:0px;padding-top:var(--global-kb-spacing-sm, 1.5rem);padding-right:var(--global-kb-spacing-sm, 1.5rem);padding-bottom:var(--global-kb-spacing-sm, 1.5rem);padding-left:var(--global-kb-spacing-sm, 1.5rem);}.kt-accordion-id_cc7480-e4 > .kt-accordion-inner-wrap > .wp-block-kadence-pane > .kt-accordion-header-wrap > .kt-blocks-accordion-header{border-top-color:#555555;border-right-color:#555555;border-bottom-color:#555555;border-left-color:#555555;border-top-width:0px;border-right-width:0px;border-bottom-width:0px;border-left-width:0px;border-top-left-radius:0px;border-top-right-radius:0px;border-bottom-right-radius:0px;border-bottom-left-radius:0px;background:#f2f2f2;font-size:18px;line-height:24px;color:#555555;padding-top:10px;padding-right:14px;padding-bottom:10px;padding-left:14px;}.kt-accordion-id_cc7480-e4:not( .kt-accodion-icon-style-basiccircle ):not( .kt-accodion-icon-style-xclosecircle ):not( .kt-accodion-icon-style-arrowcircle )  > .kt-accordion-inner-wrap > .wp-block-kadence-pane > .kt-accordion-header-wrap .kt-blocks-accordion-icon-trigger:after, .kt-accordion-id_cc7480-e4:not( .kt-accodion-icon-style-basiccircle ):not( .kt-accodion-icon-style-xclosecircle ):not( .kt-accodion-icon-style-arrowcircle )  > .kt-accordion-inner-wrap > .wp-block-kadence-pane > .kt-accordion-header-wrap .kt-blocks-accordion-icon-trigger:before{background:#555555;}.kt-accordion-id_cc7480-e4:not( .kt-accodion-icon-style-basic ):not( .kt-accodion-icon-style-xclose ):not( .kt-accodion-icon-style-arrow ) .kt-blocks-accordion-icon-trigger{background:#555555;}.kt-accordion-id_cc7480-e4:not( .kt-accodion-icon-style-basic ):not( .kt-accodion-icon-style-xclose ):not( .kt-accodion-icon-style-arrow ) .kt-blocks-accordion-icon-trigger:after, .kt-accordion-id_cc7480-e4:not( .kt-accodion-icon-style-basic ):not( .kt-accodion-icon-style-xclose ):not( .kt-accodion-icon-style-arrow ) .kt-blocks-accordion-icon-trigger:before{background:#f2f2f2;}.kt-accordion-id_cc7480-e4 > .kt-accordion-inner-wrap > .wp-block-kadence-pane > .kt-accordion-header-wrap > .kt-blocks-accordion-header:hover, \n\t\t\t\tbody:not(.hide-focus-outline) .kt-accordion-id_cc7480-e4 .kt-blocks-accordion-header:focus-visible{color:#444444;background:#eeeeee;border-top-color:#eeeeee;border-right-color:#eeeeee;border-bottom-color:#eeeeee;border-left-color:#eeeeee;}.kt-accordion-id_cc7480-e4:not( .kt-accodion-icon-style-basiccircle ):not( .kt-accodion-icon-style-xclosecircle ):not( .kt-accodion-icon-style-arrowcircle ) .kt-accordion-header-wrap .kt-blocks-accordion-header:hover .kt-blocks-accordion-icon-trigger:after, .kt-accordion-id_cc7480-e4:not( .kt-accodion-icon-style-basiccircle ):not( .kt-accodion-icon-style-xclosecircle ):not( .kt-accodion-icon-style-arrowcircle ) .kt-accordion-header-wrap .kt-blocks-accordion-header:hover .kt-blocks-accordion-icon-trigger:before, body:not(.hide-focus-outline) .kt-accordion-id_cc7480-e4:not( .kt-accodion-icon-style-basiccircle ):not( .kt-accodion-icon-style-xclosecircle ):not( .kt-accodion-icon-style-arrowcircle ) .kt-blocks-accordion--visible .kt-blocks-accordion-icon-trigger:after, body:not(.hide-focus-outline) .kt-accordion-id_cc7480-e4:not( .kt-accodion-icon-style-basiccircle ):not( .kt-accodion-icon-style-xclosecircle ):not( .kt-accodion-icon-style-arrowcircle ) .kt-blocks-accordion-header:focus-visible .kt-blocks-accordion-icon-trigger:before{background:#444444;}.kt-accordion-id_cc7480-e4:not( .kt-accodion-icon-style-basic ):not( .kt-accodion-icon-style-xclose ):not( .kt-accodion-icon-style-arrow ) .kt-accordion-header-wrap .kt-blocks-accordion-header:hover .kt-blocks-accordion-icon-trigger, body:not(.hide-focus-outline) .kt-accordion-id_cc7480-e4:not( .kt-accodion-icon-style-basic ):not( .kt-accodion-icon-style-xclose ):not( .kt-accodion-icon-style-arrow ) .kt-accordion-header-wrap .kt-blocks-accordion-header:focus-visible .kt-blocks-accordion-icon-trigger{background:#444444;}.kt-accordion-id_cc7480-e4:not( .kt-accodion-icon-style-basic ):not( .kt-accodion-icon-style-xclose ):not( .kt-accodion-icon-style-arrow ) .kt-accordion-header-wrap .kt-blocks-accordion-header:hover .kt-blocks-accordion-icon-trigger:after, .kt-accordion-id_cc7480-e4:not( .kt-accodion-icon-style-basic ):not( .kt-accodion-icon-style-xclose ):not( .kt-accodion-icon-style-arrow ) .kt-accordion-header-wrap .kt-blocks-accordion-header:hover .kt-blocks-accordion-icon-trigger:before, body:not(.hide-focus-outline) .kt-accordion-id_cc7480-e4:not( .kt-accodion-icon-style-basic ):not( .kt-accodion-icon-style-xclose ):not( .kt-accodion-icon-style-arrow ) .kt-accordion-header-wrap .kt-blocks-accordion-header:focus-visible .kt-blocks-accordion-icon-trigger:after, body:not(.hide-focus-outline) .kt-accordion-id_cc7480-e4:not( .kt-accodion-icon-style-basic ):not( .kt-accodion-icon-style-xclose ):not( .kt-accodion-icon-style-arrow ) .kt-accordion-header-wrap .kt-blocks-accordion-header:focus-visible .kt-blocks-accordion-icon-trigger:before{background:#eeeeee;}.kt-accordion-id_cc7480-e4 .kt-accordion-header-wrap .kt-blocks-accordion-header:focus-visible,\n\t\t\t\t.kt-accordion-id_cc7480-e4 > .kt-accordion-inner-wrap > .wp-block-kadence-pane > .kt-accordion-header-wrap > .kt-blocks-accordion-header.kt-accordion-panel-active{color:#ffffff;background:#444444;border-top-color:#444444;border-right-color:#444444;border-bottom-color:#444444;border-left-color:#444444;}.kt-accordion-id_cc7480-e4:not( .kt-accodion-icon-style-basiccircle ):not( .kt-accodion-icon-style-xclosecircle ):not( .kt-accodion-icon-style-arrowcircle )  > .kt-accordion-inner-wrap > .wp-block-kadence-pane > .kt-accordion-header-wrap > .kt-blocks-accordion-header.kt-accordion-panel-active .kt-blocks-accordion-icon-trigger:after, .kt-accordion-id_cc7480-e4:not( .kt-accodion-icon-style-basiccircle ):not( .kt-accodion-icon-style-xclosecircle ):not( .kt-accodion-icon-style-arrowcircle )  > .kt-accordion-inner-wrap > .wp-block-kadence-pane > .kt-accordion-header-wrap > .kt-blocks-accordion-header.kt-accordion-panel-active .kt-blocks-accordion-icon-trigger:before{background:#ffffff;}.kt-accordion-id_cc7480-e4:not( .kt-accodion-icon-style-basic ):not( .kt-accodion-icon-style-xclose ):not( .kt-accodion-icon-style-arrow ) .kt-blocks-accordion-header.kt-accordion-panel-active .kt-blocks-accordion-icon-trigger{background:#ffffff;}.kt-accordion-id_cc7480-e4:not( .kt-accodion-icon-style-basic ):not( .kt-accodion-icon-style-xclose ):not( .kt-accodion-icon-style-arrow ) .kt-blocks-accordion-header.kt-accordion-panel-active .kt-blocks-accordion-icon-trigger:after, .kt-accordion-id_cc7480-e4:not( .kt-accodion-icon-style-basic ):not( .kt-accodion-icon-style-xclose ):not( .kt-accodion-icon-style-arrow ) .kt-blocks-accordion-header.kt-accordion-panel-active .kt-blocks-accordion-icon-trigger:before{background:#444444;}@media all and (max-width: 767px){.kt-accordion-id_cc7480-e4 .kt-accordion-inner-wrap{display:block;}.kt-accordion-id_cc7480-e4 .kt-accordion-inner-wrap .kt-accordion-pane:not(:first-child){margin-top:0px;}}<\/style>\n<div class=\"wp-block-kadence-accordion alignnone\"><div class=\"kt-accordion-wrap kt-accordion-wrap kt-accordion-id_cc7480-e4 kt-accordion-has-2-panes kt-active-pane-0 kt-accordion-block kt-pane-header-alignment-left kt-accodion-icon-style-basic kt-accodion-icon-side-right\" style=\"max-width:none\"><div class=\"kt-accordion-inner-wrap\" data-allow-multiple-open=\"true\" data-start-open=\"none\">\n<div class=\"wp-block-kadence-pane kt-accordion-pane kt-accordion-pane-1 kt-pane_0d2a17-9f\"><div class=\"kt-accordion-header-wrap\"><button class=\"kt-blocks-accordion-header kt-acccordion-button-label-show\"><div class=\"kt-blocks-accordion-title-wrap\"><span class=\"kt-blocks-accordion-title\">Definition of Orthogonal Projection<\/span><\/div><div class=\"kt-blocks-accordion-icon-trigger\"><\/div><\/button><\/div><div class=\"kt-accordion-panel kt-accordion-panel-hidden\"><div class=\"kt-accordion-panel-inner\">\n<p>\n\n According to the Projection Theorem, if $W$ is a finite dimensional subspace of an inner product space $V$ , then every vector ${\\bf u} \\in V$ can be expressed uniquely as $$ {\\bf u} = {\\bf w}_1 + {\\bf w}_2, $$ where ${\\bf w}_1 \\in W$ and ${\\bf w}_2$ is orthogonal to $W$. The vector ${\\bf w}_1$ is called the <strong>orthogonal projection of ${\\bf u}$ on $W$<\/strong> and is denoted by ${\\bf w}_1 = \\textrm{proj}_{W} {\\bf u}$.  \n\n<\/p>\n<\/div><\/div><\/div>\n<\/div><\/div><\/div>\n\n\n\n<p><br> <u>Step 3<\/u> Let ${\\bf v}_3 = {\\bf u}_3 &#8211; \\mbox{proj}_{W_{2}} {\\bf u}_3 = {\\bf u}_3- \\frac{\\langle {\\bf u}_3,{\\bf v}_1\\rangle}{\\| {\\bf v}_1 \\|^{2}}{\\bf v}_1 &#8211; \\frac{\\langle {\\bf u}_3,{\\bf v}_2\\rangle}{\\| {\\bf v}_2 \\|^{2}}{\\bf v}_2$ where $W_2$ is the space spanned by ${\\bf v}_1$ and ${\\bf v}_2$. <\/p>\n\n\n\n<p>\n<u>Step 4<\/u> Let ${\\bf v}_4 = {\\bf u}_4 &#8211;\n\t\t\\mbox{proj}_{W_{3}} {\\bf u}_4 = {\\bf u}_4- \\frac{\\langle\n\t\t{\\bf u}_4,{\\bf v}_1\\rangle}{\\| {\\bf v}_1 \\|^{2}}{\\bf v}_1 &#8211; \\frac{\\langle\n\t\t{\\bf u}_4,{\\bf v}_2\\rangle}{\\| {\\bf v}_2 \\|^{2}}{\\bf v}_2 &#8211; \\frac{\\langle\n\t\t{\\bf u}_4,{\\bf v}_3\\rangle}{\\| {\\bf v}_3 \\|^{2}}{\\bf v}_3$ where $W_3$ is\n\t\tthe space spanned by ${\\bf v}_1, {\\bf v}_2$ and ${\\bf v}_3$.\n<\/p>\n\n\n\n<p>\n&nbsp; &nbsp; &nbsp; $\\vdots$\n\n\n<\/p>\n\n\n\n<p>\nContinue this process up to ${\\bf v}_n$.  The resulting orthogonal set\n$\\left\\{{\\bf v}_1,{\\bf v}_2,\\ldots,{\\bf v}_n\\right\\}$ consists of $n$ linearly\nindependent vectors in $V$ and so forms an orthogonal basis for $V$.\n\n<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"220\" height=\"220\" src=\"https:\/\/i0.wp.com\/math.hmc.edu\/calculus\/wp-content\/uploads\/sites\/3\/2019\/08\/gramschm.gif?resize=220%2C220&#038;ssl=1\" alt=\"The Gram-Schmidt Algorithm creating an orthogonal set of vectors v.\" class=\"wp-image-355\"\/><\/figure><\/div>\n\n\n\n<center>\n\n<br><br><br><br><br>\n<h4>Notes<\/h4>\n<\/center>\n\n\n\n<ul class=\"wp-block-list\"><li> To obtain an <b>orthonormal<\/b> basis,  which is an orthogonal set in which each vector has norm 1, for an inner product space $V$, use the Gram-Schmidt algorithm to construct an orthogonal basis. Then simply normalize each vector in the basis. <br><br> <\/li><li> For $R^{n}$ with the Eudlidean inner product (dot product), we of course already know of the orthonormal basis $\\left\\{  (1,0,0,\\ldots,0), (0,1,0,\\ldots,0), \\ldots , (0, \\ldots, 0,1)\\right\\}$.  For more abstract spaces, however, the existence of an orthonormal basis is not obvious.  The Gram-Schmidt algorithm is powerful in that it not only guarantees the existence of an orthonormal basis for any inner product space, but actually gives the construction of such a basis. <\/li><\/ul>\n\n\n\n<h6 class=\"wp-block-heading\">Example<\/h6>\n\n\n\n<p>\n\nLet $V=R^{3}$ with the Euclidean inner product.  We will apply the\nGram-Schmidt algorithm to orthogonalize the basis $\\left\\{ \n(1,-1,1),(1,0,1),(1,1,2)\\right\\}$.\n\n<\/p>\n\n\n\n<p>\n<u>Step 1<\/u> ${\\bf v}_1 = (1,-1,1)$.\n<\/p>\n\n\n\n<p>\n<u>Step 2<\/u> $\\begin{array}{rcl}\n\t\t\t{\\bf v}_2 &amp; = &amp; (1,0,1) &#8211; \\frac{(1,0,1) \\cdot\n\t\t\t(1,-1,1)}{\\|(1,-1,1)\\|^{2}}(1,-1,1)\\\\\n\t\t\t&amp; = &amp; (1,0,1) &#8211; \\frac{2}{3}(1,-1,1)\\\\\n\t\t\t&amp; = &amp; (\\frac{1}{3},\\frac{2}{3},\\frac{1}{3}).\n\t\t\\end{array}$\n<\/p>\n\n\n\n<p>\n<u>Step 3<\/u> $\\begin{array}{rcl}\n\t\t\t{\\bf v}_3 &amp; = &amp; (1,1,2) &#8211; \\frac{(1,1,2) \\cdot (1,-1,1)}{\\|(1,-1,1)\\|^{2}}(1,-1,1)\n\t\t\t\t &#8211; \\frac{(1,1,2) \\cdot (\\frac{1}{3},\\frac{2}{3},\\frac{1}{3}) \\strut}{\\|(\\frac{1}{3},\\frac{2}{3},\\frac{1}{3})\\|^{2}}\n\t\t\t\t \t(\\frac{1}{3},\\frac{2}{3},\\frac{1}{3}) \\\\\n\t\t\t&amp; = &amp; (1,1,2) &#8211; \\frac{2}{3}(1,-1,1)-\\frac{5}{2}(\\frac{1}{3},\\frac{2}{3},\\frac{1}{3})\\\\\n\t\t\t&amp; = &amp; (\\frac{-1}{2},0,\\frac{1}{2}).\n\t\t\\end{array}$\n\n\n<\/p>\n\n\n\n<p>\nYou can verify that $\\left\\{ \n(1,-1,1),(\\frac{1}{3},\\frac{2}{3},\\frac{1}{3}),(\\frac{-1}{2},0,\\frac{1}{2})\\right\\}$ forms an orthogonal\nbasis for $R^{3}$.  Normalizing the vectors in the orthogonal basis,\nwe obtain the orthonormal basis\n$$\n\\left\\{ \\left( \\frac{\\sqrt{3}}{3}, \\frac{-\\sqrt{3}}{3},\n\\frac{\\sqrt{3}}{3}\\right), \\left( \\frac{\\sqrt{6}}{6},\n\\frac{\\sqrt{6}}{3}, \\frac{\\sqrt{6}}{6}\\right), \\left\n( \\frac{-\\sqrt{2}}{2}, 0, \\frac{\\sqrt{2}}{2}\\right) \\right\\}.\n$$\n\n<\/p>\n\n\n\n<hr class=\"wp-block-separator\"\/>\n\n\n\n<center>\n<h4>Key Concepts<\/h4>\n<\/center>\n\n\n\n<p>\n\nGiven an arbitrary basis $\\left\\{ {\\bf u}_1,{\\bf u}_2,\\ldots,{\\bf u}_n\\right\\}$\nfor an $n$-dimensional inner product space $V$, the <\/p>\n\n\n\n<b>Gram-Schmidt\nalgorithm<\/b>\n\n\n\n<p> constructs an orthogonal basis $\\left\\{ {\\bf\nv}_1,{\\bf v}_2,\\ldots,{\\bf v}_n\\right\\}$ for $V$:\n\n<\/p>\n\n\n\n<p>\n<u>Step 1<\/u> Let ${\\bf v}_1 = {\\bf u}_1$.\n<\/p>\n\n\n\n<p>\n<u>Step 2<\/u> Let ${\\bf v}_2 = {\\bf u}_2 &#8211; \\frac{\\langle\n\t\t{\\bf u}_2,{\\bf v}_1\\rangle}{\\| {\\bf v}_1 \\|^{2}}{\\bf v}_1$.\n<\/p>\n\n\n\n<p>\n<u>Step 3<\/u> Let ${\\bf v}_3 = {\\bf u}_3- \\frac{\\langle\n\t\t{\\bf u}_3,{\\bf v}_1\\rangle}{\\| {\\bf v}_1 \\|^{2}}{\\bf v}_1 &#8211; \\frac{\\langle\n\t\t{\\bf u}_3,{\\bf v}_2\\rangle}{\\| {\\bf v}_2 \\|^{2}}{\\bf v}_2$. \n<\/p>\n\n\n\n<p>\n&nbsp; &nbsp; &nbsp; $\\vdots$\n\n\n\n\n<!------------------------>\n\n\n<br>\n\n<\/p>\n\n\n\n<hr class=\"wp-block-separator\"\/>\n\n\n\n<p>\n\n[<a href=\"https:\/\/physics.hmc.edu\/ct\/quiz\/QZ3710\/\">I&#8217;m ready to take the quiz.<\/a>]\n[<a href=\"#top\">I need to review more.<\/a>]<br>\n\n\n<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The Gram-Schmidt Algorithm &#8211; HMC Calculus Tutorial In any inner product space, we can choose the basis in which to work. It often greatly simplifies calculations to work in an orthogonal basis. For one thing, if $ S = \\{ {\\bf v}_1, {\\bf v}_2, \\dots , {\\bf v}_n \\} $ is an orthogonal basis for&hellip;<\/p>\n","protected":false},"author":5,"featured_media":0,"parent":61,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"tags":[],"class_list":["post-227","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/math.hmc.edu\/calculus\/wp-json\/wp\/v2\/pages\/227","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/math.hmc.edu\/calculus\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/math.hmc.edu\/calculus\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/math.hmc.edu\/calculus\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/math.hmc.edu\/calculus\/wp-json\/wp\/v2\/comments?post=227"}],"version-history":[{"count":6,"href":"https:\/\/math.hmc.edu\/calculus\/wp-json\/wp\/v2\/pages\/227\/revisions"}],"predecessor-version":[{"id":1236,"href":"https:\/\/math.hmc.edu\/calculus\/wp-json\/wp\/v2\/pages\/227\/revisions\/1236"}],"up":[{"embeddable":true,"href":"https:\/\/math.hmc.edu\/calculus\/wp-json\/wp\/v2\/pages\/61"}],"wp:attachment":[{"href":"https:\/\/math.hmc.edu\/calculus\/wp-json\/wp\/v2\/media?parent=227"}],"wp:term":[{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/math.hmc.edu\/calculus\/wp-json\/wp\/v2\/tags?post=227"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}