Integrity
Write
Loading...
Will Lockett

Will Lockett

3 years ago

The World Will Change With MIT's New Battery

More on Technology

Gajus Kuizinas

Gajus Kuizinas

3 years ago

How a few lines of code were able to eliminate a few million queries from the database

I was entering tens of millions of records per hour when I first published Slonik PostgreSQL client for Node.js. The data being entered was usually flat, making it straightforward to use INSERT INTO ... SELECT * FROM unnset() pattern. I advocated the unnest approach for inserting rows in groups (that was part I).

Bulk inserting nested data into the database

However, today I’ve found a better way: jsonb_to_recordset.

jsonb_to_recordset expands the top-level JSON array of objects to a set of rows having the composite type defined by an AS clause.

jsonb_to_recordset allows us to query and insert records from arbitrary JSON, like unnest. Since we're giving JSON to PostgreSQL instead of unnest, the final format is more expressive and powerful.

SELECT *
FROM json_to_recordset('[{"name":"John","tags":["foo","bar"]},{"name":"Jane","tags":["baz"]}]')
AS t1(name text, tags text[]);
 name |   tags
------+-----------
 John | {foo,bar}
 Jane | {baz}
(2 rows)

Let’s demonstrate how you would use it to insert data.

Inserting data using json_to_recordset

Say you need to insert a list of people with attributes into the database.

const persons = [
  {
    name: 'John',
    tags: ['foo', 'bar']
  },
  {
    name: 'Jane',
    tags: ['baz']
  }
];

You may be tempted to traverse through the array and insert each record separately, e.g.

for (const person of persons) {
  await pool.query(sql`
    INSERT INTO person (name, tags)
    VALUES (
      ${person.name},
      ${sql.array(person.tags, 'text[]')}
    )
  `);
}

It's easier to read and grasp when working with a few records. If you're like me and troubleshoot a 2M+ insert query per day, batching inserts may be beneficial.

What prompted the search for better alternatives.

Inserting using unnest pattern might look like this:

await pool.query(sql`
  INSERT INTO public.person (name, tags)
  SELECT t1.name, t1.tags::text[]
  FROM unnest(
    ${sql.array(['John', 'Jane'], 'text')},
    ${sql.array(['{foo,bar}', '{baz}'], 'text')}
  ) AS t1.(name, tags);
`);

You must convert arrays into PostgreSQL array strings and provide them as text arguments, which is unsightly. Iterating the array to create slices for each column is likewise unattractive.

However, with jsonb_to_recordset, we can:

await pool.query(sql`
  INSERT INTO person (name, tags)
  SELECT *
  FROM jsonb_to_recordset(${sql.jsonb(persons)}) AS t(name text, tags text[])
`);

In contrast to the unnest approach, using jsonb_to_recordset we can easily insert complex nested data structures, and we can pass the original JSON document to the query without needing to manipulate it.

In terms of performance they are also exactly the same. As such, my current recommendation is to prefer jsonb_to_recordset whenever inserting lots of rows or nested data structures.

Thomas Smith

2 years ago

ChatGPT Is Experiencing a Lightbulb Moment

Why breakthrough technologies must be accessible

ChatGPT has exploded. Over 1 million people have used the app, and coding sites like Stack Overflow have banned its answers. It's huge.

I wouldn't have called that as an AI researcher. ChatGPT uses the same GPT-3 technology that's been around for over two years.

More than impressive technology, ChatGPT 3 shows how access makes breakthroughs usable. OpenAI has finally made people realize the power of AI by packaging GPT-3 for normal users.

We think of Thomas Edison as the inventor of the lightbulb, not because he invented it, but because he popularized it.

Going forward, AI companies that make using AI easy will thrive.

Use-case importance

Most modern AI systems use massive language models. These language models are trained on 6,000+ years of human text.

GPT-3 ate 8 billion pages, almost every book, and Wikipedia. It created an AI that can write sea shanties and solve coding problems.

Nothing new. I began beta testing GPT-3 in 2020, but the system's basics date back further.

Tools like GPT-3 are hidden in many apps. Many of the AI writing assistants on this platform are just wrappers around GPT-3.

Lots of online utilitarian text, like restaurant menu summaries or city guides, is written by AI systems like GPT-3. You've probably read GPT-3 without knowing it.

Accessibility

Why is ChatGPT so popular if the technology is old?

ChatGPT makes the technology accessible. Free to use, people can sign up and text with the chatbot daily. ChatGPT isn't revolutionary. It does it in a way normal people can access and be amazed by.

Accessibility isn't easy. OpenAI's Sam Altman tweeted that opening ChatGPT to the public increased computing costs.

Each chat costs "low-digit cents" to process. OpenAI probably spends several hundred thousand dollars a day to keep ChatGPT running, with no immediate business case.

Academic researchers and others who developed GPT-3 couldn't afford it. Without resources to make technology accessible, it can't be used.

Retrospective

This dynamic is old. In the history of science, a researcher with a breakthrough idea was often overshadowed by an entrepreneur or visionary who made it accessible to the public.

We think of Thomas Edison as the inventor of the lightbulb. But really, Vasilij Petrov, Thomas Wright, and Joseph Swan invented the lightbulb. Edison made technology visible and accessible by electrifying public buildings, building power plants, and wiring.

Edison probably lost a ton of money on stunts like building a power plant to light JP Morgan's home, the NYSE, and several newspaper headquarters.

People wanted electric lights once they saw their benefits. By making the technology accessible and visible, Edison unlocked a hugely profitable market.

Similar things are happening in AI. ChatGPT shows that developing breakthrough technology in the lab or on B2B servers won't change the culture.

AI must engage people's imaginations to become mainstream. Before the tech impacts the world, people must play with it and see its revolutionary power.

As the field evolves, companies that make the technology widely available, even at great cost, will succeed.

OpenAI's compute fees are eye-watering. Revolutions are costly.

Nikhil Vemu

Nikhil Vemu

3 years ago

7 Mac Tips You Never Knew You Needed

Unleash the power of the Option key ⌥

Photo by Michał Kubalczyk on Unsplash

#1 Open a link in the Private tab first.

Previously, if I needed to open a Safari link in a private window, I would:

  • copied the URL with the right click command,

  • choose File > New Private Window to open a private window, and

  • clicked return after pasting the URL.

I've found a more straightforward way.

Right-clicking a link shows this, right?

This, and all the images below are by the author

Hold option (⌥) for:

‘Open Link in New Private Window’ in Mac Safari

Click Open Link in New Private Window while holding.

Finished!

#2. Instead of searching for specific characters, try this

You may use unicode for business or school. Most people Google them when they need them.

That is lengthy!

You can type some special characters just by pressing ⌥ and a key.

For instance

• ⌥+2 -> ™ (Trademark)
• ⌥+0 -> ° (Degree)
• ⌥+G -> © (Copyright)
• ⌥+= -> ≠ (Not equal to)
• ⌥+< -> ≤ (Less than or equal to)
• ⌥+> -> ≥ (Greater then or equal to)
• ⌥+/ -> ÷ (Different symbol for division)

#3 Activate Do Not Disturb silently.

Do Not Disturb when sharing my screen is awkward for me (because people may think Im trying to hide some secret notifications).

Here's another method.

Hold ⌥ and click on Time (at the extreme right on the menu-bar).

Menubar in Mac

Now, DND is activated (secretly!). To turn it off, do it again.

Note: This works only for DND focus.

#4. Resize a window starting from its center

Although this is rarely useful, it is still a hidden trick.

When you resize a window, the opposite edge or corner is used as the pivot, right?

However, if you want to resize it with its center as the pivot, hold while doing so.

#5. Yes, Cut-Paste is available on Macs as well (though it is slightly different).

I call it copy-move rather than cut-paste. This is how it works.

Carry it out.

Choose a file (by clicking on it), then copy it (+C).

Go to a new location on your Mac. Do you use +V to paste it? However, to move it, press ⌘+⌥+V.

This removes the file from its original location and copies it here. And it works exactly like cut-and-paste on Windows.

#6. Instantly expand all folders

Set your Mac's folders to List view.

Assume you have one folder with multiple subfolders, each of which contains multiple files. And you wanted to look at every single file that was over there.

How would you do?

You're used to clicking the ⌄ glyph near the folder and each subfolder to expand them all, right? Instead, hold down ⌥ while clicking ⌄ on the parent folder.

This is what happens next.

Everything expands.

View/Copy a file's path as an added bonus

If you want to see the path of a file in Finder, select it and hold ⌥, and you'll see it at the bottom for a moment.

To copy its path, right-click on the folder and hold down ⌥ to see this

Click on Copy <"folder name"> as Pathname to do it.

#7 "Save As"

I was irritated by the lack of "Save As" in Pages when I first got a Mac (after 15 years of being a Windows guy).

It was necessary for me to save the file as a new file, in a different location, with a different name, or both.

Unfortunately, I couldn't do it on a Mac.

However, I recently discovered that it appears when you hold ⌥ when in the File menu.

Yay!

You might also like

Logan Rane

Logan Rane

2 years ago

I questioned Chat-GPT for advice on the top nonfiction books. Here's What It Suggests

You have to use it.

Chat-GPT Logo

Chat-GPT is a revolution.

All social media outlets are discussing it. How it will impact the future and different things.

True.

I've been using Chat-GPT for a few days, and it's a rare revolution. It's amazing and will only improve.

I asked Chat-GPT about the best non-fiction books. It advised this, albeit results rely on interests.

The Immortal Life of Henrietta Lacks

by Rebecca Skloot

Science, Biography

A impoverished tobacco farmer dies of cervical cancer in The Immortal Life of Henrietta Lacks. Her cell strand helped scientists treat polio and other ailments.

Rebecca Skloot discovers about Henrietta, her family, how the medical business exploited black Americans, and how her cells can live forever in a fascinating and surprising research.

You ought to read it.

  1. if you want to discover more about the past of medicine.

  2. if you want to discover more about American history.

Bad Blood: Secrets and Lies in a Silicon Valley Startup

by John Carreyrou

Tech, Bio

Bad Blood tells the terrifying story of how a Silicon Valley tech startup's blood-testing device placed millions of lives at risk.

John Carreyrou, a Pulitzer Prize-winning journalist, wrote this book.

Theranos and its wunderkind CEO, Elizabeth Holmes, climbed to popularity swiftly and then plummeted.

You ought to read it.

  1. if you are a start-up employee.

  2. specialists in medicine.

The Power of Now: A Guide to Spiritual Enlightenment

by Eckhart Tolle

Self-improvement, Spirituality

The Power of Now shows how to stop suffering and attain inner peace by focusing on the now and ignoring your mind.

The book also helps you get rid of your ego, which tries to control your ideas and actions.

If you do this, you may embrace the present, reduce discomfort, strengthen relationships, and live a better life.

You ought to read it.

  1. if you're looking for serenity and illumination.

  2. If you believe that you are ruining your life, stop.

  3. if you're not happy.

The 7 Habits of Highly Effective People

by Stephen R. Covey

Profession, Success

The 7 Habits of Highly Effective People is an iconic self-help book.

This vital book offers practical guidance for personal and professional success.

This non-fiction book is one of the most popular ever.

You ought to read it.

  1. if you want to reach your full potential.

  2. if you want to discover how to achieve all your objectives.

  3. if you are just beginning your journey toward personal improvement.

Sapiens: A Brief History of Humankind

by Yuval Noah Harari

Science, History

Sapiens explains how our species has evolved from our earliest ancestors to the technology age.

How did we, a species of hairless apes without tails, come to control the whole planet?

It describes the shifts that propelled Homo sapiens to the top.

You ought to read it.

  1. if you're interested in discovering our species' past.

  2. if you want to discover more about the origins of human society and culture.

Claire Berehova

Claire Berehova

3 years ago

There’s no manual for that

Kyiv oblast in springtime. Photo by author.

We’ve been receiving since the war began text messages from the State Emergency Service of Ukraine every few days. They’ve contained information on how to comfort a child and what to do in case of a water outage.

But a question that I struggle to suppress irks within me: How would we know if there really was a threat coming our away? So how can I happily disregard an air raid siren and continue singing to my three-month-old son when I feel like a World War II film became reality? There’s no manual for that.

Along with the anxiety, there’s the guilt that always seems to appear alongside dinner we’re fortunate to still have each evening while brave Ukrainian soldiers are facing serious food insecurity. There’s no manual for how to deal with this guilt.

When it comes to the enemy, there is no manual for how to react to the news of Russian casualties. Every dead Russian soldier weakens Putin, but I also know that many of these men had wives and girlfriends who are now living a nightmare.

So, I felt like I had to start writing my own manual.

The anxiety around the air raid siren? Only with time does it get easier to ignore it, but never completely.

The guilt? All we can do is pray.

That inner conflict? As Russia continues to stun the world with its war crimes, my emotions get less gray — I have to get used to accommodating absurd levels of hatred.

Sadness? It feels a bit more manageable when we laugh, and a little alcohol helps (as it usually does).

Cabin fever? Step outside in the yard when possible. At least the sunshine is becoming more fervent with spring approaching.

Slava Ukraini. Heroyam slava. (Glory to Ukraine. Glory to the heroes.)

Quant Galore

Quant Galore

3 years ago

I created BAW-IV Trading because I was short on money.

More retail traders means faster, more sophisticated, and more successful methods.

Tech specifications

Only requires a laptop and an internet connection.

We'll use OpenBB's research platform for data/analysis.

OpenBB

Pricing and execution on Options-Quant

Options-Quant

Background

You don't need to know the arithmetic details to use this method.

Black-Scholes is a popular option pricing model. It's best for pricing European options. European options are only exercisable at expiration, unlike American options. American options are always exercisable.

American options carry a premium to cover for the risk of early exercise. The Black-Scholes model doesn't account for this premium, hence it can't price genuine, traded American options.

Barone-Adesi-Whaley (BAW) model. BAW modifies Black-Scholes. It accounts for exercise risk premium and stock dividends. It adds the option's early exercise value to the Black-Scholes value.

The trader need not know the formulaic derivations of this model.

https://ir.nctu.edu.tw/bitstream/11536/14182/1/000264318900005.pdf

Strategy

This strategy targets implied volatility. First, we'll locate liquid options that expire within 30 days and have minimal implied volatility.

After selecting the option that meets the requirements, we price it to get the BAW implied volatility (we choose BAW because it's a more accurate Black-Scholes model). If estimated implied volatility is larger than market volatility, we'll capture the spread.

(Calculated IV — Market IV) = (Profit)

Some approaches to target implied volatility are pricey and inaccessible to individual investors. The best and most cost-effective alternative is to acquire a straddle and delta hedge. This may sound terrifying and pricey, but as shown below, it's much less so.

The Trade

First, we want to find our ideal option, so we use OpenBB terminal to screen for options that:

  • Have an IV at least 5% lower than the 20-day historical IV

  • Are no more than 5% out-of-the-money

  • Expire in less than 30 days

We query:

stocks/options/screen/set low_IV/scr --export Output.csv

This uses the screener function to screen for options that satisfy the above criteria, which we specify in the low IV preset (more on custom presets here). It then saves the matching results to a csv(Excel) file for viewing and analysis.

Stick to liquid names like SPY, AAPL, and QQQ since getting out of a position is just as crucial as getting in. Smaller, illiquid names have higher inefficiencies, which could restrict total profits.

Output of option screen (Only using AAPL/SPY for liquidity)

We calculate IV using the BAWbisection model (the bisection is a method of calculating IV, more can be found here.) We price the IV first.

Parameters for Pricing IV of Call Option; Interest Rate = 30Day T-Bill RateOutput of Implied Volatilities

According to the BAW model, implied volatility at this level should be priced at 26.90%. When re-pricing the put, IV is 24.34%, up 3%.

Now it's evident. We must purchase the straddle (long the call and long the put) assuming the computed implied volatility is more appropriate and efficient than the market's. We just want to speculate on volatility, not price fluctuations, thus we delta hedge.

The Fun Starts

We buy both options for $7.65. (x100 multiplier). Initial delta is 2. For every dollar the stock price swings up or down, our position value moves $2.

Initial Position Delta

We want delta to be 0 to avoid price vulnerability. A delta of 0 suggests our position's value won't change from underlying price changes. Being delta-hedged allows us to profit/lose from implied volatility. Shorting 2 shares makes us delta-neutral.

Delta After Shorting 2 Shares

That's delta hedging. (Share price * shares traded) = $330.7 to become delta-neutral. You may have noted that delta is not truly 0.00. This is common since delta-hedging means getting as near to 0 as feasible, since it is rare for deltas to align at 0.00.

Now we're vulnerable to changes in Vega (and Gamma, but given we're dynamically hedging, it's not a big risk), or implied volatility. We wanted to gamble that the position's IV would climb by at least 2%, so we'll maintain it delta-hedged and watch IV.

Because the underlying moves continually, the option's delta moves continuously. A trader can short/long 5 AAPL shares at most. Paper trading lets you practice delta-hedging. Being quick-footed will help with this tactic.

Profit-Closing

As expected, implied volatility rose. By 10 minutes before market closure, the call's implied vol rose to 27% and the put's to 24%. This allowed us to sell the call for $4.95 and the put for $4.35, creating a profit of $165.

You may pull historical data to see how this trade performed. Note the implied volatility and pricing in the final options chain for August 5, 2022 (the position date).

Call IV of 27%, Put IV of 24%

Final Thoughts

Congratulations, that was a doozy. To reiterate, we identified tickers prone to increased implied volatility by screening OpenBB's low IV setting. We double-checked the IV by plugging the price into Options-BAW Quant's model. When volatility was off, we bought a straddle and delta-hedged it. Finally, implied volatility returned to a normal level, and we profited on the spread.

The retail trading space is very quickly catching up to that of institutions.  Commissions and fees used to kill this method, but now they cost less than $5. Watching momentum, technical analysis, and now quantitative strategies evolve is intriguing.

I'm not linked with these sites and receive no financial benefit from my writing.

Tell me how your experience goes and how I helped; I love success tales.