I was entering tens of millions of records per hour when I first published Slonik PostgreSQL client for Node.js. The data being entered was usually flat, making it straightforward to use INSERT INTO ... SELECT * FROM unnset() pattern. I advocated the unnest approach for inserting rows in groups (that was part I).

However, today I’ve found a better way: jsonb_to_recordset.

jsonb_to_recordset expands the top-level JSON array of objects to a set of rows having the composite type defined by an AS clause.

jsonb_to_recordset allows us to query and insert records from arbitrary JSON, like unnest. Since we're giving JSON to PostgreSQL instead of unnest, the final format is more expressive and powerful.

SELECT *
FROM json_to_recordset('[{"name":"John","tags":["foo","bar"]},{"name":"Jane","tags":["baz"]}]')
AS t1(name text, tags text[]);
 name |   tags
------+-----------
 John | {foo,bar}
 Jane | {baz}
(2 rows)

Let’s demonstrate how you would use it to insert data.

Inserting data using json_to_recordset

Say you need to insert a list of people with attributes into the database.

const persons = [
  {
    name: 'John',
    tags: ['foo', 'bar']
  },
  {
    name: 'Jane',
    tags: ['baz']
  }
];

You may be tempted to traverse through the array and insert each record separately, e.g.

for (const person of persons) {
  await pool.query(sql`
    INSERT INTO person (name, tags)
    VALUES (
      ${person.name},
      ${sql.array(person.tags, 'text[]')}
    )
  `);
}

It's easier to read and grasp when working with a few records. If you're like me and troubleshoot a 2M+ insert query per day, batching inserts may be beneficial.

What prompted the search for better alternatives.

Inserting using unnest pattern might look like this:

await pool.query(sql`
  INSERT INTO public.person (name, tags)
  SELECT t1.name, t1.tags::text[]
  FROM unnest(
    ${sql.array(['John', 'Jane'], 'text')},
    ${sql.array(['{foo,bar}', '{baz}'], 'text')}
  ) AS t1.(name, tags);
`);

You must convert arrays into PostgreSQL array strings and provide them as text arguments, which is unsightly. Iterating the array to create slices for each column is likewise unattractive.

However, with jsonb_to_recordset, we can:

await pool.query(sql`
  INSERT INTO person (name, tags)
  SELECT *
  FROM jsonb_to_recordset(${sql.jsonb(persons)}) AS t(name text, tags text[])
`);

In contrast to the unnest approach, using jsonb_to_recordset we can easily insert complex nested data structures, and we can pass the original JSON document to the query without needing to manipulate it.

In terms of performance they are also exactly the same. As such, my current recommendation is to prefer jsonb_to_recordset whenever inserting lots of rows or nested data structures.

A few days ago, I was discussing dall-e with two art and tech pals. One artist acquaintance said she knew a frightened illustrator. Would the ability to create anything with a click derail her career? The artist feared this. My curator friend smiled and said this has always been a dread among artists. When the camera was invented, didn't painters say this? Even in the Instagram era, painting exists.

When art and technology collide, there's room for innovation, experimentation, and fear — especially if the technology replicates or replaces art making. What is art's future with dall-e? How does technology affect music, beyond visual art? Recently, I saw "ABBA Voyage," a holographic ABBA concert in London.

"Abba voyage?" my phone asked in early March. A Gen X friend I met through a fashion blogging ring texted me.

"What's abba Voyage?" I asked while opening my front door with keys and coffee.

We're going! Marti, visiting London, took me to a show.

"Absolutely no ABBA songs here." I responded.

My parents didn't play ABBA much, so I don't know much about them. Dad liked Jimi Hendrix, Cream, Deep Purple, and New Orleans jazz. Marti told me ABBA Voyage was a holographic ABBA show with a live band.

The show was fun, extraordinary fun. Nearly everyone on the dance floor wore wigs, ankle-breaking platforms, sequins, and bellbottoms. I saw some millennials and Zoomers among the boomers.

I was intoxicated by the experience.

Automatons date back to the 18th-century mechanical turk. The mechanical turk was a chess automaton operated by a person. The mechanical turk seemed to perform like a human without human intervention, but it required a human in the loop to work properly.

Humans have used non-humans in entertainment for centuries, such as puppets, shadow play, and smoke and mirrors. A show can have animatronic, technological, and non-technological elements, and a live show can blur real and illusion. From medieval puppet shows to mechanical turks to AI filters, bots, and holograms, entertainment has evolved over time.

I'm not a hologram skeptic, but I'm skeptical of technology, especially since I work with it. I love live performances, I love hearing singers breathe, forget lines, and make jokes. Live shows are my favorite because I love watching performers make mistakes or interact with the audience. ABBA Voyage was different.

Marti and I traveled to Manchester after ABBA Voyage to see Liam Gallagher. Similar but different vibe. Similar in that thousands dressed up for the show. ABBA's energy was dizzying. 90s chic replaced sequins in the crowd. Doc Martens, nylon jackets, bucket hats, shaggy hair. The Charlatans and Liam Gallagher opened and closed, respectively. Fireworks. Incredible. People went crazy. Yelling exhausted my voice.

This week in music featured AI-enabled holograms and a decades-old rocker. Both are warm and gooey in our memories.

After seeing both, I'm wondering if we need AI hologram shows. Why? Is it good?

Like everything tech-related, my answer is "maybe." Because context and performance matter. Liam Gallagher and ABBA both had great, different shows.

For a hologram to work, it must be impossible and big. It must be big, showy, and improbable to justify a hologram. It must feel...expensive, like a stadium pop show. According to a quick search, ABBA broke up on bad terms. Reuniting is unlikely. This is also why Prince or Tupac hologram shows work. We can only engage with their legacy through covers or...holograms.

I drove around listening to the radio a few weeks ago. "Dreaming of You" by Selena played. Selena's music defined my childhood. I sang along and turned up the volume (or as loud as my husband would allow me while driving on the highway).

I discovered Selena's music six months after her death, so I never saw her perform live. My babysitter Melissa played me her album after I moved to Houston. Melissa took me to see the Selena movie five times when it came out. I quickly wore out my VHS copy. I constantly sang "Bibi Bibi Bom Bom" and "Como la Flor." I love Selena. A Selena hologram? Yes, probably.

Instagram advertised a cellist's Arthur Russell tribute show. Russell is another deceased artist I love. I almost walked down the aisle to "This is How We Walk on the Moon," but our cellist couldn't find it. Instead, I walked to Magnetic Fields' "The Book of Love." I "discovered" Russell after a friend introduced me to his music a few years ago.

I use these as analogies for the Liam Gallagher and ABBA concerts.

You have no idea how much I'd pay to see a hologram of Selena's 1995 Houston Livestock Show and Rodeo concert. Arthur Russell's hologram is unnecessary. Russell's work was intimate and performance-based. We can't separate his life from his legacy; popular audiences overlooked his genius. He died of AIDS broke. Like Selena, he died prematurely. Given his music and history, another performer would be a better choice than a hologram. He's no Selena. Selena could have rivaled Beyonce.

Pop shows' size works for holograms. Along with ABBA holograms, there was an anime movie and a light show that would put Tron to shame. ABBA created a tourable stadium show. The event was lavish, expensive, and well-planned. Pop, unlike rock, isn't gritty. Liam Gallagher hologram? No longer impossible, it wouldn't work. He's touring. I'm not sure if a rockstar alone should be rendered as a hologram; it was the show that made ABBA a hologram.

Holograms, like AI, are part of the future of entertainment, but not all of it. Because only modern interpretations of Arthur Russell's work reveal his legacy. That's his legacy.

Large-scale arena performers may use holograms in the future, but the experience must be impossible. A teacher once said that the only way to convey emotion in opera is through song, and I feel the same way about holograms, AR, VR, and mixed reality. A story's impossibility must make sense, like in opera. Impossibility and bombastic performance must be present for an immersive element to "work." ABBA was an impossible and improbable experience, which made it magical. It helped the holographic show work.

Marti told me about ABBA Voyage. She said it was a great concert. Marti has worked in music since the 1990s. She's a music expert; she's seen many shows.

Ai isn't a god or sentient, and the ABBA holograms aren't real. The renderings were glassy-eyed, flat, and robotic, like the Polar Express or the Jaws shark. Even today, the uncanny valley is insurmountable. We know it's not real because it's not about reality. It was about a suspended moment and performance feelings.

I knew this was impossible, an 'unreal' experience, but the emotions I felt were real, like watching a movie or tv show. Perhaps this is one of the better uses of AI, like CGI and special effects, like the beauty of entertainment- we were enraptured and entertained for hours. I've been playing ABBA since then.

Starting a passive income stream with data science and programming

My website is fresh. But how do I monetize it?

Creating a passive-income website is difficult. Advertise first. But what useful are ads without traffic?

Let’s Generate Traffic And Put Our Programming Skills To Use

SEO boosts traffic (Search Engine Optimisation). Traffic generation is complex. Keywords matter more than text, URL, photos, etc.

My Python skills helped here. I wanted to find relevant, Google-trending keywords (tags) for my topic.

First The Code

I wrote the script below here.

import re
from string import punctuation

import nltk
from nltk import TreebankWordTokenizer, sent_tokenize
from nltk.corpus import stopwords


class KeywordsGenerator:
    def __init__(self, pytrends):
        self._pytrends = pytrends

    def generate_tags(self, file_path, top_words=30):
        file_text = self._get_file_contents(file_path)
        clean_text = self._remove_noise(file_text)
        top_words = self._get_top_words(clean_text, top_words)
        suggestions = []
        for top_word in top_words:
            suggestions.extend(self.get_suggestions(top_word))
        suggestions.extend(top_words)
        tags = self._clean_tokens(suggestions)
        return ",".join(list(set(tags)))

    def _remove_noise(self, text):
        #1. Convert Text To Lowercase and remove numbers
        lower_case_text = str.lower(text)
        just_text = re.sub(r'\d+', '', lower_case_text)
        #2. Tokenise Paragraphs To words
        list = sent_tokenize(just_text)
        tokenizer = TreebankWordTokenizer()
        tokens = tokenizer.tokenize(just_text)
        #3. Clean text
        clean = self._clean_tokens(tokens)
        return clean

    def _clean_tokens(self, tokens):
        clean_words = [w for w in tokens if w not in punctuation]
        stopwords_to_remove = stopwords.words('english')
        clean = [w for w in clean_words if w not in stopwords_to_remove and not w.isnumeric()]
        return clean

    def get_suggestions(self, keyword):
        print(f'Searching pytrends for {keyword}')
        result = []
        self._pytrends.build_payload([keyword], cat=0, timeframe='today 12-m')
        data = self._pytrends.related_queries()[keyword]['top']
        if data is None or data.values is None:
            return result
        result.extend([x[0] for x in data.values.tolist()][:2])
        return result

    def _get_file_contents(self, file_path):
        return open(file_path, "r", encoding='utf-8',errors='ignore').read()

    def _get_top_words(self, words, top):
        counts = dict()

        for word in words:
            if word in counts:
                counts[word] += 1
            else:
                counts[word] = 1

        return list({k: v for k, v in sorted(counts.items(), key=lambda item: item[1])}.keys())[:top]


if __name__ == "1__main__":
    from pytrends.request import TrendReq

    nltk.download('punkt')
    nltk.download('stopwords')
    pytrends = TrendReq(hl='en-GB', tz=360)
    tags = KeywordsGenerator(pytrends)\
              .generate_tags('text_file.txt')
    print(tags)

Then The Dependencies

This script requires:

nltk==3.7
pytrends==4.8.0

Analysis of the Script

I copy and paste my article into text file.txt, and the code returns the keywords as a comma-separated string.

To achieve this:

1. A class I made is called KeywordsGenerator.

2. This class has a function: generate_tags

3. The function generate_tags performs the following tasks:

• retrieves text file contents

• uses NLP to clean the text by tokenizing sentences into words, removing punctuation, and other elements.

• identifies the most frequent words that are relevant.

• The pytrends API is then used to retrieve related phrases that are trending for each word from Google.

• finally adds a comma to the end of the word list.

4. I then use the keywords and paste them into the SEO area of my website.

These terms are trending on Google and relevant to my topic. My site's rankings and traffic have improved since I added new keywords. This little script puts our knowledge to work. I shared the script in case anyone faces similar issues.

I hope it helps readers sell their work.

An inspiring reading list

I'm no billionaire or hedge-fund manager. My bank account doesn't have millions. Ray Dalio's love of reading motivates me to think differently.

Here are some books recommended by Ray Dalio. Each influenced me. Hope they'll help you.

Sapiens by Yuval Noah Harari

Page Count: 512
Rating on Goodreads: 4.39

My favorite nonfiction book.

Sapiens explores human evolution. It explains how Homo Sapiens developed from hunter-gatherers to a dominant species. Amazing!

Sapiens will teach you about human history. Yuval Noah Harari has a follow-up book on human evolution.

My favorite book quotes are:

• The tendency for luxuries to turn into necessities and give rise to new obligations is one of history's few unbreakable laws.

• Happiness is not dependent on material wealth, physical health, or even community. Instead, it depends on how closely subjective expectations and objective circumstances align.

• The romantic comparison between today's industry, which obliterates the environment, and our forefathers, who coexisted well with nature, is unfounded. Homo sapiens held the record among all organisms for eradicating the most plant and animal species even before the Industrial Revolution. The unfortunate distinction of being the most lethal species in the history of life belongs to us.

The Power Of Habit by Charles Duhigg

Page Count: 375
Rating on Goodreads: 4.13

Great book: The Power Of Habit. It illustrates why habits are everything. The book explains how healthier habits can improve your life, career, and society.

The Power of Habit rocks. It's a great book on productivity. Its suggestions helped me build healthier behaviors (and drop bad ones).

Read ASAP!

My favorite book quotes are:

• Change may not occur quickly or without difficulty. However, almost any behavior may be changed with enough time and effort.

• People who exercise begin to eat better and produce more at work. They are less smokers and are more patient with friends and family. They claim to feel less anxious and use their credit cards less frequently. A fundamental habit that sparks broad change is exercise.

• Habits are strong but also delicate. They may develop independently of our awareness or may be purposefully created. They frequently happen without our consent, but they can be altered by changing their constituent pieces. They have a much greater influence on how we live than we realize; in fact, they are so powerful that they cause our brains to adhere to them above all else, including common sense.

Tribe Of Mentors by Tim Ferriss

Page Count: 561
Rating on Goodreads: 4.06

Unusual book structure. It's worth reading if you want to learn from successful people.

The book is Q&A-style. Tim questions everyone. Each chapter features a different person's life-changing advice. In the book, Pressfield, Willink, Grylls, and Ravikant are interviewed.

Amazing!

My favorite book quotes are:

• According to one's courage, life can either get smaller or bigger.

• Don't engage in actions that you are aware are immoral. The reputation you have with yourself is all that constitutes self-esteem. Always be aware.

• People mistakenly believe that focusing means accepting the task at hand. However, that is in no way what it represents. It entails rejecting the numerous other worthwhile suggestions that exist. You must choose wisely. Actually, I'm just as proud of the things we haven't accomplished as I am of what I have. Saying no to 1,000 things is what innovation is.

His success was unforeseeable.

2014 saw Facebook's $2.3 billion purchase of Oculus VR.

19-year-old Palmer Luckey founded Oculus. He quit journalism school. His parents worried about his college dropout.

Facebook bought Oculus VR in less than 2 years.

Palmer Luckey started Anduril Industries. Palmer has raised $385 million with Anduril.

The Oculus journey began in a trailer

Palmer Luckey, 19, owned the trailer.

Luckey had his trailer customized. The trailer had all six of Luckey's screens. In the trailer's remaining area, Luckey conducted hardware tests.

At 16, he became obsessed with virtual reality. Virtual reality was rare at the time.

Luckey didn't know about VR when he started.

Previously, he liked "portabilizing" mods. Hacking ancient game consoles into handhelds.

In his city, fewer portabilizers actively traded.

Luckey started "ModRetro" for other portabilizers. Luckey was exposed to VR headsets online.

Luckey:

“Man, ModRetro days were the best.”

Palmer Luckey used VR headsets for three years. His design had 50 prototypes.

Luckey used to work at the Long Beach Sailing Center for minimum salary, servicing diesel engines and cleaning boats.

Luckey worked in a USC Institute for Creative Technologies mixed reality lab in July 2011. (ICT).

Luckey cleaned the lab, did reports, and helped other students with VR projects.

Luckey's lab job was dull.

Luckey chose to work in the lab because he wanted to engage with like-minded folks.

By 2012, Luckey had a prototype he hoped to share globally. He made cheaper headsets than others.

Luckey wanted to sell an easy-to-assemble virtual reality kit on Kickstarter.

He realized he needed a corporation to do these sales legally. He started looking for names. "Virtuality," "virtual," and "VR" are all taken.

Hence, Oculus.

If Luckey sold a hundred prototypes, he would be thrilled since it would boost his future possibilities.

John Carmack, legendary game designer

Carmack has liked sci-fi and fantasy since infancy.

Carmack loved imagining intricate gaming worlds.

His interest in programming and computer science grew with age.

He liked graphics. He liked how mismatching 0 and 1 might create new colors and visuals.

Carmack played computer games as a teen. He created Shadowforge in high school.

He founded Id software in 1991. When Carmack created id software, console games were the best-sellers.

Old computer games have weak graphics. John Carmack and id software developed "adaptive tile refresh."

This technique smoothed PC game scrolling. id software launched 3-D, Quake, and Doom using "adaptive tile refresh."

These games made John Carmack a gaming star. Later, he sold Id software to ZeniMax Media.

How Palmer Luckey met Carmack

In 2011, Carmack was thinking a lot about 3-D space and virtual reality.

He was underwhelmed by the greatest HMD on the market. Because of their flimsiness and latency.

His disappointment was partly due to the view (FOV). Best HMD had 40-degree field of view.

Poor. The best VR headset is useless with a 40-degree FOV.

Carmack intended to show the press Doom 3 in VR. He explored VR headsets and internet groups for this reason.

Carmack identified a VR enthusiast in the comments section of "LEEP on the Cheap." "PalmerTech" was the name.

Carmack approached PalmerTech about his prototype. He told Luckey about his VR demos, so he wanted to see his prototype.

Carmack got a Rift prototype. Here's his May 17 tweet.

John Carmack tweeted an evaluation of the Luckey prototype.

Dan Newell, a Valve engineer, and Mick Hocking, a Sony senior director, pre-ordered Oculus Rift prototypes with Carmack's help.

Everyone praised Luckey after Carmack demoed Rift.

Palmer Luckey received a job offer from Sony.

• It was a full-time position at Sony Computer Europe.

• He would run Sony’s R&D lab.

• The salary would be $70k.

Who is Brendan Iribe?

Brendan Iribe started early with Startups. In 2004, he and Mike Antonov founded Scaleform.

Scaleform created high-performance middleware. This package allows 3D Flash games.

In 2011, Iribe sold Scaleform to Autodesk for $36 million.

How Brendan Iribe discovered Palmer Luckey.

Brendan Iribe's friend Laurent Scallie.

Laurent told Iribe about a potential opportunity.

Laurent promised Iribe VR will work this time. Laurent introduced Iribe to Luckey.

Iribe was doubtful after hearing Laurent's statements. He doubted Laurent's VR claims.

But since Laurent took the name John Carmack, Iribe thought he should look at Luckey Innovation. Iribe was hooked on virtual reality after reading Palmer Luckey stories.

He asked Scallie about Palmer Luckey.

Iribe convinced Luckey to start Oculus with him

First meeting between Palmer Luckey and Iribe.

The Iribe team wanted Luckey to feel comfortable.

Iribe sought to convince Luckey that launching a company was easy. Iribe told Luckey anyone could start a business.

Luckey told Iribe's staff he was homeschooled from childhood. Luckey took self-study courses.

Luckey had planned to launch a Kickstarter campaign and sell kits for his prototype. Many companies offered him jobs, nevertheless.

He's considering Sony's offer.

Iribe advised Luckey to stay independent and not join a firm. Iribe asked Luckey how he could raise his child better. No one sees your baby like you do?

Iribe's team pushed Luckey to stay independent and establish a software ecosystem around his device.

After conversing with Iribe, Luckey rejected every job offer and merger option.

Iribe convinced Luckey to provide an SDK for Oculus developers.

After a few months. Brendan Iribe co-founded Oculus with Palmer Luckey. Luckey trusted Iribe and his crew, so he started a corporation with him.

Crowdfunding

Brendan Iribe and Palmer Luckey launched a Kickstarter.

Gabe Newell endorsed Palmer's Kickstarter video.

Gabe Newell wants folks to trust Palmer Luckey since he's doing something fascinating and answering tough questions.

Mark Bolas and David Helgason backed Palmer Luckey's VR Kickstarter video.

Luckey introduced Oculus Rift during the Kickstarter campaign. He introduced virtual reality during press conferences.

Oculus' Kickstarter effort was a success. Palmer Luckey felt he could raise $250,000.

Oculus raised $2.4 million through Kickstarter. Palmer Luckey's virtual reality vision was well-received.

Mark Zuckerberg's Oculus discovery

Brendan Iribe and Palmer Luckey hired the right personnel after a successful Kickstarter campaign.

Oculus needs a lot of money for engineers and hardware. They needed investors' money.

Series A raised $16M.

Next, Andreessen Horowitz partner Brain Cho approached Iribe.

Cho told Iribe that Andreessen Horowitz could invest in Oculus Series B if the company solved motion sickness.

Mark Andreessen was Iribe's dream client.

Marc Andreessen and his partners gave Oculus $75 million.

Andreessen introduced Iribe to Zukerberg. Iribe and Zukerberg discussed the future of games and virtual reality by phone.

Facebook's Oculus demo

Iribe showed Zuckerberg Oculus.

Mark was hooked after using Oculus. The headset impressed him.

The whole Facebook crew who saw the demo said only one thing.

“Holy Crap!”

This surprised them all.

Mark Zuckerberg was impressed by the team's response. Mark Zuckerberg met the Oculus team five days after the demo.

First meeting Palmer Luckey.

Palmer Luckey is one of Mark's biggest supporters and loves Facebook.

Oculus Acquisition

Zuckerberg wanted Oculus.

Brendan Iribe had requested for $4 billion, but Mark wasn't interested.

Facebook bought Oculus for $2.3 billion after months of drama.

After selling his company, how does Palmer view money?

Palmer loves the freedom money gives him. Money frees him from small worries.

Money has allowed him to pursue things he wouldn't have otherwise.

“If I didn’t have money I wouldn’t have a collection of vintage military vehicles…You can have nice hobbies that keep you relaxed when you have money.”

He didn't start Oculus to generate money. His virtual reality passion spanned years.

He didn't have to lie about how virtual reality will transform everything until he needed funding.

The company's success was an unexpected bonus. He was merely passionate about a good cause.

After Oculus' $2.3 billion exit, what changed?

Palmer didn't mind being rich. He did similar things.

After Facebook bought Oculus, he moved to Silicon Valley and lived in a 12-person shared house due to high rents.

Palmer might have afforded a big mansion, but he prefers stability and doing things because he wants to, not because he has to.

“Taco Bell is never tasted so good as when you know you could afford to never eat taco bell again.”

Palmer's leadership shifted.

Palmer changed his leadership after selling Oculus.

When he launched his second company, he couldn't work on his passions.

“When you start a tech company you do it because you want to work on a technology, that is why you are interested in that space in the first place. As the company has grown, he has realized that if he is still doing optical design in the company it’s because he is being negligent about the hiring process.”

Once his startup grows, the founder's responsibilities shift. He must recruit better firm managers.

Recruiting talented people becomes the top priority. The founder must convince others of their influence.

A book that helped me write this:

The History of the Future: Oculus, Facebook, and the Revolution That Swept Virtual Reality — Blake Harris

*This post is a summary. Read the full article here.