kb autocommit

content/posts/KBhinner_product.md

@@ -63,7 +63,7 @@ similar to [dot product]({{< relref "KBhdot_product.md" >}}) for the reals. This
 
 ...as both relevant and more general than the [dot product]({{< relref "KBhdot_product.md" >}}), but also different in key areas.
 
-[First, review complex numbers]({{< relref "KBhthoughts_on_axler_4.md#first-review-complex-number--kbhcomplex-number-dot-md--s" >}}) from our discussion in chapter 4. The main problem here is this:
+[First, review complex numbers]({{< relref "KBhthoughts_on_axler_4.md#first-review-id-7c982e7e-b8be-4053-a71e-fc0dba7a5da5-complex-number-s" >}}) from our discussion in chapter 4. The main problem here is this:
 
 for \\(z = (z\_1, \dots, z\_{n}) \in \mathbb{C}^{n}\\), simply squaring each slot to take the [norm]({{< relref "KBhnorm.md" >}}) may cause us to take a square root of a negative number (as each slot would then be \\(a^{2}-b^{2}\\) for a complex number). That's no bueno because we want \\(\\|z\\|\\) to be real and non-negative.
 
@@ -80,3 +80,20 @@ x \cdot y = \bar{{y \cdot x}}
 \end{equation}
 
 derived by following the usual rules of [complex conjugation]({{< relref "KBhcomplex_number.md#complex-conjugate" >}}). Note that none of these elementwisethings (the \\(x\_{n}y\_{n}\\) business) are actually in the definition of the [inner product]({{< relref "KBhinner_product.md" >}}), as it is the rules of an [Euclidean Inner Product](#euclidean-inner-product).
+
+
+### inner product of \\(L\\) periodic functions {#inner-product-of-l-periodic-functions}
+
+For \\(f,g : [0,L] \to \mathbb{R}\\), which are [L-periodic]({{< relref "KBhsu_math53_feb252024.md#l-periodicity" >}}), we define:
+
+\begin{equation}
+\langle f,g \rangle := \frac{1}{L} \int\_{0}^{L} f(x) g(x) \dd{x}
+\end{equation}
+
+Recall that [L-periodic]({{< relref "KBhsu_math53_feb252024.md#l-periodicity" >}}) functions can be shifted without changing periodicity. But if for some reason you want to base it off of any two numbers with distance \\(L\\) in between:
+
+\begin{equation}
+\langle f,g \rangle\_{[a,b]} := \frac{1}{b-a} \int^{b}\_{a} f(x) g(x) \dd{x}
+\end{equation}
+
+The work of checking this is a well-formed [inner product]({{< relref "KBhinner_product.md" >}}) is left to absolutely nobody.

content/posts/KBhis_despot.md

@@ -6,7 +6,7 @@ draft = false
 
 ## Motivation {#motivation}
 
-Large crowd navigation with sudden changes: unlikely events are out of likely sample. So, we want to bring in another distribution based on **importance** and not **likeliness**.
+Large crowd navigation with sudden changes: unlikely events are out of likely sample. So, we want to bring in another distribution based on **importance** and not **likelyness**.
 
 
 ## Goals {#goals}
@@ -15,33 +15,61 @@ Large crowd navigation with sudden changes: unlikely events are out of likely sa
 -   outperforms [DESPOT]({{< relref "KBhdespot.md" >}}) and [POMCP]({{< relref "KBhpomcp.md" >}})
 
 
-## Importance Sampling {#importance-sampling}
+## [DESPOT]({{< relref "KBhdespot.md" >}}) with [Importance Sampling](#importance-sampling) {#despot--kbhdespot-dot-md--with-importance-sampling--org7062454}
 
-If you want to compute the expected value of a variable \\(u\\) that is not integrable that well, you have to sample points and average them to estimate the expected value.
+1.  take our initial belief
+2.  sample trajectories according to [Importance Sampling](#importance-sampling) distribution
+3.  calculate values of those states
+4.  obtain value estimate based on weighted average of the values
 
-So, there's a distribution over \\(f\\):
+
+### [Importance Sampling](#importance-sampling) of trajectories {#importance-sampling--org7062454--of-trajectories}
+
+We define an [importance distribution](#importance-sampling) of some trajectory \\(\xi\\):
 
 \begin{equation}
-q(s) = \frac{b(s)}{w\_{\pi}(s)}
+q(\xi | b,\pi) = q(s\_0) \prod\_{t=0}^{D} q(s\_{t+1}, o\_{t+1} | s\_{t}, a\_{t+1})
 \end{equation}
 
-where
+
+## Background {#background}
+
+
+### Importance Sampling {#importance-sampling}
+
+Suppose you have a function \\(f(s)\\) which isn't super well integrate-able, yet you want:
 
 \begin{equation}
-w(s) = \frac{\mathbb{E}\_{b} \qty( \sqrt{[\mathbb{E}(v|s, \pi )]^{2} + [Var(v|s, \pi )]})}{[\mathbb{E}(v|s, \pi )]^{2} + [Var(v|s, \pi )]}
+\mu = \mathbb{E}(f(s)) = \int\_{0}^{1} f(s)p(s) \dd{s}
 \end{equation}
 
-which measures how important a state is, where \\(\pi\\) is the total discounted reward.
+how would you sample various \\(f(s)\\) effectively such that you end up with \\(\hat{\mu}\\) that's close enough?
 
+Well, what if you have an [importance distribution](#importance-sampling) \\(q(s): S \to \mathbb{R}^{[0,1]}\\), which tells you how "important" to the expected value of the distribution a particular state is? Then, we can formulate a new, better normalization function called the "[importance weight](#importance-sampling)":
 
-## [DESPOT]({{< relref "KBhdespot.md" >}}) with [Importance Sampling](#importance-sampling) {#despot--kbhdespot-dot-md--with-importance-sampling--orgb52f65b}
+\begin{equation}
+w(s) = \frac{p(s)}{q(s)}
+\end{equation}
 
-1.  begin with states from belief
-2.  sample according to [Importance Sampling](#importance-sampling) distribution
-3.  calculate values of those states
-4.  obtain estimate based on weighted average
+Therefore, this would make our estimator:
 
+\begin{equation}
+\hat{\mu} = \frac{\sum\_{n} f(s\_{n}) w(s\_{n})}{\sum\_{n}  w(s\_{n})}
+\end{equation}
+
+
+#### Theoretic grantees {#theoretic-grantees}
+
+So, there's a distribution over \\(f\\):
 
-### Constructing \\(w\\) {#constructing-w}
+\begin{equation}
+q(s) = \frac{b(s)}{w\_{\pi}(s)}
+\end{equation}
 
-To address expectation and variance of the importance distribution, we need to collect states
+where
+
+\begin{equation}
+w(s) = \frac{\mathbb{E}\_{b} \qty( \sqrt{[\mathbb{E}(v|s, \pi )]^{2} + [Var(v|s, \pi )]})}{[\mathbb{E}(v|s, \pi )]^{2} + [Var(v|s, \pi )]}
+\end{equation}
+
+which measures how important a state is, where \\(\pi\\) is the total discounted reward.

content/posts/KBhlearn_more.md

@@ -0,0 +1,41 @@
++++
+title = "Fourier formula"
+author = ["Houjun Liu"]
+draft = false
++++
+
+For vector \\(v\\) in the [span]({{< relref "KBhspan.md" >}}) of [orthogonal]({{< relref "KBhorthogonal.md" >}}) basis \\(v\_1, ..v\_{n}\\):
+
+\begin{equation}
+v = c\_1 v\_1 + \dots + c\_{n} v\_{n}
+\end{equation}
+
+we can write:
+
+\begin{equation}
+c\_{j} = \frac{v \cdot v\_{j}}{ v\_{j} \cdot v\_{j}}
+\end{equation}
+
+---
+
+Proof:
+
+\begin{equation}
+\langle v, v\_{j} \rangle = c\_{n} \langle v\_{1}, v\_{j} \rangle \dots
+\end{equation}
+
+which is \\(0\\) for all cases that's not \\(\langle v\_{j}, v\_{j} \rangle\\) as the \\(v\\) are orthogonal, and \\(\mid v\_{j} \mid^{2}\\) for the case where it is.
+
+Hence, we see that:
+
+\begin{equation}
+\langle v, v\_{j} \rangle = c\_{j} \mid v\_{j}\mid^{2}
+\end{equation}
+
+Which gives:
+
+\begin{equation}
+c\_{j} = \frac{\langle v,v\_{j} \rangle}{\mid v\_{j}\mid^{2}}
+\end{equation}
+
+as desired.

content/posts/KBhmomdp.md

@@ -26,20 +26,47 @@ Solving the algorithm uses [SARSOP]({{< relref "KBhsarsop.md" >}}), or any point
 
 ### True Mixed Observability {#true-mixed-observability}
 
-Go about splitting about your space based on the true observability part. Say there are \\(10\\) states which are observable, you literally just initialize 10 sets of [alpha vector]({{< relref "KBhalpha_vector.md" >}})s to create:
+Go about splitting about your space based on the true observability part. Say there are \\(10\\) states which are observable, you literally just initialize 10 sets of [alpha vector]({{< relref "KBhalpha_vector.md" >}})s to create \\(V\_{1} ... V\_{10}\\) for your observable states, where each one you have:
 
 \begin{equation}
-V(x, b\_{y}) = \dots
+V\_{x\_{i}}(b\_{j}) = \dots
 \end{equation}
 
-whereby all of your objectives and backup, etc., takes \\(x\\) your observable state as input. Then, during inference/backup looking at where you are.
+whereby all of your objectives and backup, etc., takes \\(x\\) your observable state as input. Then, during inference/backup looking at where you are in the observable part and use the value function from that part.
 
 
 ### Pseudo-Full Observability {#pseudo-full-observability}
 
-Train a fully observable model, and then use [belief]({{< relref "KBhbelief.md" >}})-weighted average during inference. This is where [QMDP]({{< relref "KBhqmdp.md" >}}) came from
+Train a fully observable model, and then use [belief]({{< relref "KBhbelief.md" >}})-weighted average during inference. This is where [QMDP]({{< relref "KBhqmdp.md" >}}) came from.
 
 
 ### Bounded Uncertainty {#bounded-uncertainty}
 
-Throw away extra uncertainty, and hence leave only the region around your observed location.
+Most of the time neither of the top two cases apply cleanly. Instead, most frequently your uncertainty in your observation is _bounded_ by a significant degree.
+
+
+#### Condition {#condition}
+
+For instance, your GPS maybe uncertain, but if it says you are in Kansas you are not in Shanghai. Formally, for \\(h: O \to S\\) (the hypothetical "preimage" of any observation), we expect that:
+
+\begin{equation}
+\frac{h(o)}{S} = c
+\end{equation}
+
+gives \\(c \ll 1\\).
+
+
+#### Solution {#solution}
+
+If we know we have [Bounded Uncertainty](#bounded-uncertainty), we can reparameterize our [POMDP]({{< relref "KBhpartially_observable_markov_decision_process.md" >}}) to an [MDP]({{< relref "KBhmarkov_decision_process.md" >}}) over observations (we call this \\(X\\)) plus a [POMDP]({{< relref "KBhpartially_observable_markov_decision_process.md" >}}) modeling [uncertainty]({{< relref "KBhuncertainty.md" >}}) in offsets from those observations (we call this \\(Y\\)).
+
+Whereby:
+
+\begin{equation}
+\begin{cases}
+T\_{x}(x'|x,y,a) = \sum\_{s' \in S} T(s' | (x,y),a) O(x'|s',a) \\\\
+T\_{y}(y'|x,x',y,a) = \frac{T((x',y') | (x,y),a) O((x',y')|s',a)}{T\_{x}(x'|x,y,a)}
+\end{cases}
+\end{equation}
+
+where our state space is now split into \\(s \in S = X \times Y\\) s.t. \\(s=(x,y)\\).

content/posts/KBhsu_math53_feb252024.md

@@ -43,3 +43,112 @@ Proof:
 \begin{equation}
 h(x+L) = af(x+L) + bg(x+L) = af(x) + bg(x) = h(x)
 \end{equation}
+
+
+## Fourier Series {#fourier-series}
+
+[Fourier Series](#fourier-series) and how to find them.
+
+
+### Statement {#statement}
+
+Suppose we have a function that satisfies:
+
+\begin{equation}
+f(x) = a\_0 + \sum\_{k=1}^{\infty} \qty( a\_{k} \cos(k \omega x) + b\_{k} \sin(k \omega x))
+\end{equation}
+
+recall that each \\(\cos(\dots)\\)  and \\(\sin (...)\\) are [orthogonal]({{< relref "KBhorthogonal.md" >}}), we can then use the [Fourier formula]({{< relref "KBhlearn_more.md" >}}) to figure the coefficients \\(a\_{k}\\), \\(b\_{k}\\).
+
+---
+
+Aside: why is \\(a\_0\\) also orthogonal?
+
+\begin{equation}
+a\_0 = a\_0 \cos (0 \omega x) = a\_0 \cdot 1  = a\_0
+\end{equation}
+
+---
+
+Therefore, by the [Fourier formula]({{< relref "KBhlearn_more.md" >}}), we expect that:
+
+\begin{equation}
+a\_0 = \frac{\langle f, 1 \rangle}{ \langle 1,1 \rangle} = \frac{1}{L} \int\_{0}^{L} f(x) \dd{x}
+\end{equation}
+
+\begin{equation}
+a\_{k} = \frac{\langle f, \cos (k\omega x) \rangle}{\langle \cos (k\omega x), \cos (k\omega x) \rangle} = \frac{1}{L / 2} \int\_{0}^{L} f(x) \cos (k\omega x) \dd{x}
+\end{equation}
+
+\begin{equation}
+b\_{k} = \frac{\langle f, \sin (k\omega x) \rangle}{\langle \sin (k\omega x), \sin (k\omega x) \rangle} = \frac{1}{L / 2} \int\_{0}^{L} f(x) \sin (k\omega x) \dd{x}
+\end{equation}
+
+
+### Background {#background}
+
+
+#### [Fourier formula]({{< relref "KBhlearn_more.md" >}}) {#fourier-formula--kbhlearn-more-dot-md}
+
+Consider some [orthogonal]({{< relref "KBhorthogonal.md" >}}) basis \\(v\_1, ... v\_{n}\\), recall that if we have:
+
+\begin{equation}
+v = c\_1 v\_1 + \dots + c\_{n} v\_{n}
+\end{equation}
+
+we can write:
+
+\begin{equation}
+c\_{j} = \frac{v \cdot v\_{j}}{ v\_{j} \cdot v\_{j}}
+\end{equation}
+
+(this is similar to [Writing a vector as a linear combination of orthonormal basis]({{< relref "KBhorthonormal_basis.md#writing-a-vector-as-a-linear-combination-of-orthonormal-basis" >}}), but recall the \\(v\_{j}\\) are **orthogonal** and not **orthonormal**, so we have to divide by the square of the norm of \\(v\\). [learn more]({{< relref "KBhlearn_more.md" >}}))
+
+
+#### [inner product of \\(L\\) periodic functions]({{< relref "KBhinner_product.md#inner-product-of-l-periodic-functions" >}}) {#inner-product-of-l-periodic-functions--kbhinner-product-dot-md}
+
+For \\(f,g : [0,L] \to \mathbb{R}\\), which are [L-periodic](#l-periodicity), we write:
+
+\begin{equation}
+\langle f,g \rangle := \frac{1}{L} \int\_{0}^{L} f(x) g(x) \dd{x}
+\end{equation}
+
+<!--list-separator-->
+
+-  properties worth noting
+
+    for continuous functions \\([0,L]\\) \\(g\_1, g\_2, h\_1, h\_2, g, h\\), the [inner product]({{< relref "KBhinner_product.md" >}}) rules hold, which gives
+
+    -   \\(\langle c\_1 g\_1 + c\_2 g\_2, h \rangle = c\_1 \langle g\_1, h \rangle + c\_2 \langle g\_2, h \rangle\\)
+    -   \\(\langle g, c\_1h\_1 + c\_2h\_2 \rangle = c\_1 \langle g, h\_1 \rangle + c\_2 \langle g, h\_2 \rangle\\)
+    -   \\(\langle h,g \rangle = \langle g,h \rangle\\), as the functions are over the reals
+    -   \\(\langle g,g \rangle\\) is zero only when \\(g\\) is zero
+
+<!--list-separator-->
+
+-  \\(L\\) periodic sinusoids are orthogonal
+
+    Recall that we have two basic [L-periodic](#l-periodicity) sinusoids:
+
+    -   \\(\sin \qty(\frac{2\pi k }{L}x)\\)
+    -   \\(\cos \qty(\frac{2\pi k }{L}x)\\)
+
+    Let's write:
+
+    \begin{equation}
+    \omega = \frac{2\pi}{L}
+    \end{equation}
+
+    then, for any distinct integer \\(k\_1 \neq k\_2, k\_1, k\_2 > 0\\), we see that:
+
+    \begin{equation}
+    \int\_{0}^{L} \cos (k\_1 \omega x) \cos (k\_2 \omega x) = \int\_{0}^{L} \sin (k\_1 \omega x) \sin (k\_2 \omega x) = \int\_{0}^{L} \cos (k\_1 \omega x) \sin (k\_2 \omega x) = 0
+    \end{equation}
+
+    Meaning, every pair of \\(\langle \\{\sin, \cos\\} (k\_1 \omega x), \\{\sin, \cos\\} (k\_1 \omega x) \rangle\\) are orthogonal.
+
+    Further, for the same \\(k\\),
+
+    \begin{equation}
+    \langle \cos (k\omega x), \cos (k \omega x) \rangle = \langle \sin (k\omega x), \sin (k \omega x) \rangle = \frac{1}{2}
+    \end{equation}