Prerequisite background knowledge for COGS 203A: Statistical Models for Cognitive Sciences I

The below facts should be natural to you, meaning you know them, can use them, and can show they hold.

Students who do not have this background are encouraged to consult resources such as Khan Academy Math for a refresher.


Sums

For a refresher, see Khan Academy’s Summation Notation.

Definition

Consider the sequence of integers \(i\) that starts at \(a\) and ends at \(b\): \(i=a,\ldots,b\). Suppose we want to apply some function \(f(i)\) to each element of this sequence, and then add together all the results. We could use the summation operator \(\sum_{i=a}^b\).

\[\sum_{i=a}^b f(i) = f(a) + f(a+1) + \ldots + f(b)\]

For example, suppose that we want \(i\) to take every value starting at \(a=2\) and finishing at \(b=4\), and we want to take the square of each of those values, so that \(f(i) = i^2\) and add the results together.

\[\begin{eqnarray} \sum_{i=2}^4 i^2 &=& 2^2+3^2+4^2\\ &=& 4+9+16 \\ &=& 29 \end{eqnarray}\]

Of course, the function \(f(i)\) can be complicated, and the sequence can be long. For example

\[\begin{eqnarray} \sum_{i=-10}^{10} e^{i \cos(i)} &=& e^{-10 \cos(-10)} +e^{-9 \cos(-9)} + \ldots + e^{10 \cos(10)} \end{eqnarray}\]

Factor out a constant

The summation operator works just like a regular sum. For example, consider the distributive property:

\[a \times c + b \times c = (a+b) \times c\]

The same property holds for the summation operator, any multiplicative constant can be moved out of the sum (\(c\) is a constant because it does not depend on the index \(i\)):

\[\sum_{i=a}^b c\times f(i) = c \times \sum_{i=a}^b f(i)\]

For example, notice that each element in this summation is multiplied by the same factor 10:

\[\begin{eqnarray} \sum_{i=2}^{4} 10i^2 &=& 10\sum_{i=2}^4 i^2 \\ &=& 10 \times (2^2 + 3^2 + 4^2) \\ &=& 10 \times 29 \\ &=& 290 \end{eqnarray}\]

Sum of a constant

If you add a constant \(c\) to itself \(N\) times, the result is \(N \times c\).

\[\sum_{i=1}^{N}c = Nc\]

For example: \(\sum_1^{10} 5 = 10 \times 5 = 50\).

Splitting Sums

Instead of summing over a sum, you can take the sum of the sums instead:

\[\sum_{i=a}^b \left(f(i)+g(i)\right) = \left(\sum_{i=a}^b f(i)\right) + \left(\sum_{i=a}^b g(i)\right)\]

For example,

\[\begin{eqnarray} \sum_{i=2}^4 (i^2+2i) &=& \left(\sum_{i=2}^4 i^2\right) + \left(\sum_{i=2}^4 2i\right)\\ &=& 29 + 18 \\ &=& 47 \end{eqnarray}\]

Or,

\[\begin{eqnarray} \sum_{i=a}^b (10i – e^i) &=& \left(10 \sum_{i=a}^b i \right) + \left(-\sum_{i=a}^b e^i\right)\end{eqnarray}\]


Products

Definition

Consider the sequence of integers \(i\) that starts at \(a\) and ends at \(b\): \(i=a,\ldots,b\). Suppose we want to apply some function \(f(i)\) to each element of this sequence, and then multiply all the results. We could use the product operator \(\prod_{i=a}^b\).

\[\prod_{i=a}^b f(i) = f(a) \times f(a+1) \times \ldots \times f(b)\]

For example, suppose that we want \(i\) to take every value starting at \(a=2\) and finishing at \(b=4\), and we want to take the square of each of those values, so that \(f(i) = i^2\) and multiply the results together.

\[\begin{eqnarray} \prod_{i=2}^4 i^2 &=& 2^2 \times 3^2 \times 4^2\\ &=& 4 \times 9 \times 16 \\ &=& 576 \end{eqnarray}\]

Factor out a constant

The product operator works just like a regular product. For example, consider this factorization:

\[(a \times c) \times (b \times c) = (a \times b) \times c^2\]

The same property holds for the product operator, any multiplicative constant can be moved out of the product (\(c\) is a constant because it does not depend on the index \(i\)):

\[\prod_{i=1}^N \left( c \times f(i) \right) = c^N \times \left( \prod_i f(i) \right)\]

For example, notice that each element in this product is multiplied by the same factor 10, so we can move this common factor out of the product:

\[\begin{eqnarray} \prod_{i=1}^{3} \left(10 \times i^2\right) &=& 10^3 \times \left(\prod_{i=1}^3 i^2\right) \\ &=& 10^3 \times (1^2 \times 2^2 \times 3^2)\\ &=& 10^3 \times 36 \\ &=& 36,000 \end{eqnarray}\]

Product of a constant

If you multiply a constant \(c\) with itself \(N\) times, the result is \(c^N\).

\[\prod_{i=1}^{N}c = c^N\]

For example, \[\prod_1^{3} 5 = 5^3 = 125\]

Splitting products

If there is a product inside the product operator, you can split the expression in two or more product operations:

\[\prod_i \left(f(i)\times g(i)\right) = \left(\prod_i f(i)\right) \times \left(\prod_i g(i)\right)\]

For example,

\[\begin{eqnarray} \prod_{i=2}^4 (2^i \times 3i) &=& \left(\prod_{i=2}^4 2^i\right) \times \left(\prod_{i=2}^{4} 3i\right) \\ &=& (2^2 \times 2^3 \times 2^4) \times \left( 3^3 \times \prod_{i=2}^{4} i \right) \\ &=& 2^{2+3+4} \times 27 \times (2 \times 3 \times 4)\\ &=& 2^9 \times 648 \\ &=& 331,776 \end{eqnarray}\]


Polynomials

A refresher on multiplying polynomials is in Khan Academy’s Quadratics.

Polynomial Multiplication

These are some examples of polynomial multiplication:

\[\begin{eqnarray} (x+y) \times (a+b) &=& (x+y) \times a + (x+y) \times b\\ &=& xa+ya+xb+yb\\ \end{eqnarray}\]

Square of a sum

\[\begin{eqnarray} (x+y)^2 &=& (x+y) \times (x+y)\\ &=& (x+y) \times x + (x+y) \times y\\ &=& x^2+yx+xy+y^2\\ &=& x^2+2xy+y^2\\ \end{eqnarray}\]

Square of a difference

\[\begin{eqnarray} (x-y)^2 &=& (x-y) \times (x-y)\\ &=& (x-y) \times x – (x-y) \times y\\ &=& x^2-yx-xy+y^2\\ &=& x^2-2xy+y^2\\ \end{eqnarray}\]

Difference of squares

\[\begin{eqnarray} (x-y)\times(x+y) &=& (x-y) \times x + (x-y) \times y\\ &=& x^2-yx+xy-y^2\\ &=& x^2-y^2\\ \end{eqnarray}\]


Logarithms and Exponentials

You should be comfortable recognizing and using the basic exponential and logarithmic identities. For a refresher, see Khan Academy’s intro to logarithms and intro to exponential functions.

Basic properties of logarithms and exponentials

  • \(a^1 = a\)

  • \(a^0 =1\)

  • \(\log(1)=0\)

  • \(\log(0) = -\infty\)

  • \(\log_a(a)=1\)

  • \(\log_a b =c\) means \(a^c = b\)

Three bases

  • Natural log: \(\log_e b = c\) means \(e^c = b\), where \(e \approx 2.71828\)
  • Base 10: \(\log_{10} b =c\) means \(10^c = b\), where \(c\) is called the order or order of magnitude
  • Base 2: \(\log_{2} b =c\) means \(2^c = b\), where \(c\) is how many doublings
  • We use natural log almost always
  • \(\log\) without a base is \(\log_e\) (natural logarithm)

The log of a product is the sum of the logs

  • \(\log(xy)=\log(x)+\log(y)\)

  • \(\log(x/y) = \log(x)-\log(y)\)

The exponential of a sum is the product of the exponentials

  • \(e^{x+y} = e^xe^y\)

  • \(e^{x-y} = e^x/e^y\)

An exponential to a power is the exponential of the product

  • \(\left(e^x\right)^a = e^{ax}\)

Exponentiation and logarithm on the same base cancel each other out

  • \(a^{\log_a b } = b\)

Logarithm of an exponentiation is the product of the exponent and the logarithm of the base

  • \(\log x^a = a\log x\)
  • \(\log \sqrt[\leftroot{-2}\uproot{2}k]{x} = \log x^\frac{1}{k} = \frac{1}{k}\log x\)

Changing bases

  • \(\log_b x = \log_a x / \log_a b\)
Print Friendly, PDF & Email