Snake_Byte[15] Fourier, Discrete and Fast Transformers

The frequency domain of mind (a mind, it must be stressed, is an unextended, massless, immaterial singularity) can produce an extended, spacetime domain of matter via ontological Fourier mathematics, and the two domains interact via inverse and forward Fourier transforms.

~ Dr. Cody Newman, The Ontological Self: The Ontological Mathematics of Consciousness

I am Optimus Transformer Ruler Of The AutoCorrelation Bots

First i trust everyone is safe. i haven’t written technical blog in a while so figured i would write a Snake_Byte on one of my favorite equations The Fourier Transform:

    \[\hat{f} (\xi)=\int_{-\infty}^{\infty}f(x)e^{-2\pi ix\xi}dx\]

More specifically we will be dealing with the Fast Fourier Transform which is an implementation of The Discrete Fourier Transform. The Fourier Transform operates on continuous signals and while i do believe we will have analog computing devices (again) in the future we have to operate on 0’s and 1’s at this juncture thus we have a discrete version thereof. The discrete version:

    \[F(x) &= f\f[k] &= \sum_{j=0}^{N-1} x[j]\left(e^{-2\pi i k/N}\right)^j\0 &\leq k < N\]


    \[f[k] &= f_e[k]+e^{-2\pi i k/N}f_o[k]\f[k+N/2] &= f_e[k]-e^{-2\pi i k/N}f_o[k]\]

The Discrete Fourier Transform (DFT) is a mathematical operation. The Fast Fourier Transform (FFT) is an efficient algorithm for the evaluation of that operation (actually, a family of such algorithms). However, it is easy to get these two confused. Often, one may see a phrase like “take the FFT of this sequence”, which really means to take the DFT of that sequence using the FFT algorithm to do it efficiently.

The Fourier sequence is a kernel operation for any number of transforms where the kernel is matched to the signal if possible. The Fourier Transform is a series of sin(\theta) and cos(\theta) which makes it really useful for audio and radar analysis.

For the FFT it only takes O(n\log{}n) for the sequence computation and as one would imagine this is a substantial gain. The most commonly used FFT algorithm is the Cooley-Tukey algorithm, which was named after J. W. Cooley and John Tukey. It’s a divide and conquer algorithm for the machine calculation of complex Fourier series. It breaks the DFT into smaller DFTs. Other FFT algorithms include the Rader’s algorithm, Winograd Fourier transform algorithm, Chirp Z-transform algorithm, etc. The only rub comes as a function of the delay throughput.

There have been amazing text books written on this subject and i will list them at the end of the blarg[1,2,3]

So lets get on with some code. First we do the usual houskeeping on import libraries as well as doing some majik for inline display if you are using JupyterNotebooks. Of note ffpack which is a package of Fortran subroutines for the fast Fourier transform. It includes complex, real, sine, cosine, and quarter-wave transforms. It was developed by Paul Swarztrauber of the National Center for Atmospheric Research, and is included in the general-purpose mathematical library SLATEC.

# House keeping libraries imports and inline plots:
import numpy as np
from scipy import fftpack
%matplotlib inline
import matplotlib.pyplot as pl

We now set up some signals where we create a sinusoid with a sample rate. We use linspace to set up the amplitude and signal length.

#frequency in cycles per second or Hertz
#this is equivalent to concert A

Frequency = 20 
# Sampling rate or the number of measurements per second
# This is the rate of digital audio

Sampling_Frequency = 100 

# set up the signal space:
time = np.linspace(0,2,2 * Sampling_Frequency, endpoint = False)
signal = np.sin(Frequency * 2 * np.pi * time)

Next we plot the sinusoid under consideration:

# plot the signal:
fif, ax = plt.subplots()
ax.plot(time, signal)
ax.set_xlabel('Time [seconds]')
ax.set_ylabel('Signal Amplitude')

Next we apply the Fast Fourier Transform and transform into the frequency domain:

X_Hat = fftpack.fft(signal)
Frequency_Component = fftpack.fftfreq(len(signal)) * Sampling_Frequency

We now plot the transformed sinusoid depicting the frequencies we generated:

# plot frequency components of the signal:
fig, ax = plt.subplots()
ax.stem(Frequency_Component, np.abs(X_Hat)) # absolute value of spectrum
ax.set_xlabel ('Frequency in Hertz [HZ] Of Transformed Signal')
ax.set_ylabel ('Frequency Domain (Spectrum) Magnitude')
ax.set_xlim(-Sampling_Frequency / 2, Sampling_Frequency / 2)

To note you will see two frequency components, this is because there are positive and negative (real and imaginary) components to the transform which is what we see using the stem plots as expected. This is because the kernel as mentioned before is both sin(\theta) and cos(\theta).

So something really cool happens when using the FFT. It is called the convolution theorem as well as Dual Domain Theory. Convolution in the time domain yields multiplication in the frequency domain. Mathematically, the convolution theorem states that under suitable conditions the Fourier transform of a convolution of two functions (or signals) is the poin-twise (Hadamard multiplication) product of their Fourier transforms. More generally, convolution in one domain (e.g., time domain) equals point-wise multiplication in the other domain (e.g., frequency domain).


    \[x(t)*h(t) &= y(t)\]

    \[X(f) H(f) &= Y(f)\]

So there you have it. A little taster on the powerful Fourier Transform.

Until Then,

#iwishyouwater <- Cloudbreak this past year

Muzak To Blarg by: Voyager Essential Max Ricther. Phenomenal. November is truly staggering.


[1] The Fourier Transform and Its Applications by Dr Ronald N Bracewell. i had the honor of taking the actual class at Stanford University from Professor Bracewell.

[2] The Fourier Transform and Its Applications by E. Roan Brigham. Graet book on butterfly and overlap-add derivations thereof.

[3] Adaptive Digital Signal Processing by Dr. Claude Lindquist. A phenomenal book on frequency domain signal processing and kernel analysis. A book ahead of its time. Professor Lindquist was a mentor and had a direct effect and affect on my career and the way i approach information theory.

¿Por qué haces apnea? (Why Do You Freedive?) and 9/11

Me Manifesting And Searching (photo by @clay.motus.liquidum)

I need the sea because it teaches me. I don’t know if I learn music or awareness, if it’s a single wave or its vast existence, or only its harsh voice or its shining suggestion of fishes and ships. The fact is that until I fall asleep, in some magnetic way I move in
the university of the waves.

~ Pablo Neruda

First i trust everyone is safe. Second today is a day that will live forever in most minds as a day that i call ‘When Belief Systems Run Amuck (WBSRA)”.

To those who lost loves ones on 9.11 my condolences. To those who are permanently scarred may peace eventually see its’ way unto and into you.

It is not random that one of my dear friends “JH” is relocating this very week to Charleston, SC. He was in the North tower on the 87th floor when the “WBSRA” occurred.

He is a great human. We are going to have a blast when he gets here.

Recently more than ever humans around me have been talking more and more about “stuff” they do besides “working” and inevitably as we are discussing these outside work endeavors someone mentions that i freedive.

Clay T. Talley Rappelling Underwater (photo Tanner Peterson @depthwishfreediving)

It goes something like this:

Human: Ted freedives.

Everyone: ( looking at me like i have obtained the classified UAP documents from the All-domain Anomaly Resolution Office (AARO)…)

Me: Yup.

Human: Wow really?

Me: (thinking… No i just lie about stuff like that…) Why Yes i do.

Me and @clay.motus.liquidum (photo courtesy of @depthwishfreediving)

Human: Have you seen “Deepest Breath” On Netflix?

Me: While i do not watch hardly any TV i did view it and yes it was amazing. The people in that video are the best of the best in the world and i have met one or two of them and have been coached and critiqued by some of them. i’m no where in their league.

Human: So really is it one breath? There must be some trick? How long can you hold your breath? How deep do you go?

Me: Yes, No, about 4:00+ minutes and i have broken 100ft . None of it really matters as the folks who are great, i pale in comparison, but it is a starting point. It is also not a contest with others. While there are competitions the only competitor is you and the water.

Me coming up from looking at the sand somewhere (photo courtesy of @depthwishfreediving)

Human: What is it like?

Me: It is truth. If there is a truth to life it is closer to living with the exception at least for me, what i find in surfing. Although the two are very different, the results, from to time, are the same. Complete un-adulterated FLOW STATE. The free fall is very close to getting a full tube ride. It is all enveloping it is not like meditating as some people think it is, although, there is a somnambulistic component to the free fall.

Clay Talley and Jacob Talley the day Jacob was certified (photo courtesy of @depthwishfreediving)

Human: What is freefall?

Me: That is after you get down to about 14 meters which is around 45 feet that is 3 atmospheres of pressure you start getting pulled down the elevator shaft as i call it. You can feel the water rushing by you and the sounds rushing, depending on if your hooked to a line or just letting your hand brush the dive line. Then as you go deeper it gets more enveloping.

Me: See your Mammalian Dive Reflect kicks in, your MDR, the master switch as it were. The sensors are still in our face. This is why it feels so nice to splash water on your face or take a shower. All those negative ions getting you balanced into BlueMind. There is much we have forgotten as a race. There is an excellent book called BlueMind i highly recommend.

Me, Thomas Tanner (@itstomfoolery) and My Nephew, Ty Tanner (@tytanner_) the day they got certified

(photo courtesy of @depthwishfreediving)

Human: Yea but how can you hold your breath that long?

Me: Practice. Actually it all happened in one weekend at Performance Freediving Institute (PFI). Me and my waterman commrade Clay Talley went down on a Friday and took a class after we read a book called “One Breath” and got turned onto a song called “Waves” by Bahamas. We have been training with weights in the pool for about 15 years for surfing then this freediving wonderland rabbit hole came along. We could easily get you to over a 2:30 breath hold in one day. Nothing prepared us for what was to come.

Human: I don’t see how that is possible.

Me: Well, i suppose with an attitude like that nothing is possible, but at least we gave it the benefit of the doubt and when we got certified. It was really intense due to several factors that i would have to explain later. However by far my certification was in one of the top experiences of my life and about as close to a religious experience as one could get.

My Son Thomas Tanner getting certified with Tanner Peterson Lower Left (photo courtesy of @tctjr)

Then then THE question:

Human: Why do you do it?

Me: Do what? Freedive?

Human: Yes.

This, Oh Dear Reader is where we start getting to the essence of the blog. Hope Y’all are still tuned-in to the channel.

To every sailor the gods have given a comrade
While one sleeps the other keeps watch on the bridge
When one doubts the other gives him his faith
When one falls the other discovers the oasis of ice for both of them
When my comrade loses faith I laugh confidently
When my comrade sleeps I keep watch for him
When my comrade falls I fight on for the both of us
Because to every sailor the gods have given a comrade

~ Blood Axis

Me: Well it is complicated. One of my best friends died freediving. He was one of the few people who really knew me and and happened to be the best coding computer scientist i have ever met, as well as an expert Waterman.

Me wondering what the Puppies are Thinking. (photo courtesy @clay.motus.luquidum)

Human: Oh i had no idea i am so sorry.

Me: Oh please no need for condolences. i’ll never get over it but that is a different issue. That said to answer your question i got into it because i wanted to understand how one of the smartest most fit watermen i have ever met came to his demise doing what he loved. i had to know or at least try to understand what Steven was searching for given i knew him so well.

It’s not tragic to die doing something you love.

~ Mark Foo

Me: See i kinda blew it off years ago when he kept talking about freediving and how incredible it was and in fact the last thing he said to me was he was really happy. That is really all i cared about to be honest. However it still nagged at me about there was a facet of “The Water” i hadn’t addressed.

So through a very seemingly random occurrence which to me wasn’t random at all i ran across the book “One Breath” which was a story of the death of one of the greatest freedivers of all time. So we (me and my commrade Clay Talley) finally got around to getting down so to speak and since then it has been a continual Alice in Wonderland – Down The Rabbit Hole. Besides the fact that you burn about 800-1000 calories an hour freediving it only teaches you more about yourself. It is indeed a fractal situation – the more complex – the more complex – the more simple – the more simple.

Me ascending and hanging out in Bonaire with Carlos Coste

(photo courtesy of @carloscoste1 of @blueclassroom and @freediveeexplorations)

Ted, Its all in Your Mind, Get me? Understand What I am saying?!

~ Cole Heitt

Me: Its really a super performant sport. All of the people that i have met have been physicists, composers, lawyers, bankers, technical folks and many have walked away from those careers in full pursuit of freediving. Everyone is very open, welcoming and focused. There is little to no overhead with some of the crap that you get with other sports endeavors, it is very binary. As an aside i personally want to dive with Sperm Whales.

Human: Yes but what about your friend?

Me: Well if i could have a discussion with him first i would punch him for being stupid then i would hug him, tell him i love him and say lets go for a freedive and this time i’ll be up and you be down.

Clay Talley Searching For Clay T. Talley

Me: Feel free to come down and get in the pool or we can take you down and get certified.

Human: Really?

Me: Of course and you can’t blame me for what will happen afterward.

To everyone that has ever asked me this is why i freedive. I needed to understand why and it has turned into a mirror for me.

On September 11, 2005, Steven Swenson took his last breath. However i know he was happy in fact the happiest i have ever heard him.

Recently his favorite musician past away, Jimmy Buffett. Steven’s favorite song was Son of A Son of A Sailor. He also liked “That is What Living Is to Me” and “One Particular Harbor”. I would like to think they are sharing sailing stories with each other with wind in their hair and water in their shoes.

Do me a favor if you knew him go listen to any of those songs and if you didn’t know him go listen and raise a glass to the ocean. i would also greatly appreciate it.

Mickey Talley @mich.888 Pushing Past Her Own Thoughts (photo courtesy @clay.motus.liquidum)

So there you have it folks.

Until Then,

#iwishyouwater <- Me and Present Crew Getting After it @iwishyouwater


Music To Blog By: The version of “Little Wing/ Third Stone: From The Sun off SRV archives and some Jimmy Buffett. The Tuck and Patti version of Castles Made of Sand / Little Wing is amazing as well.

Note: With Much Love to Roma, Lief and Gage.