This method scrapes websites with ChatGPT (demo with Amazon and Twitter)

In a recent article, I demonstrated how to scrape websites using ChatGPT prompts like scrape website X using Python.

But that doesn’t always work.

After scraping dozens of websites with ChatGPT, I realized that simple prompts rarely work for web scraping.

Using ChatGPT and basic HTML, we can scrape any website.

First things first, use ChatGPT's advanced version (Playground)

ChatGPT Playground's enhanced version is needed to scrape websites quickly. It generates code faster and has less constraints.

Here’s how it looks.

The classic ChatGPT display just allows you to type a prompt. Playground is more customizable and generates code faster than the basic version.

No more delays or restrictions.

In this tutorial, we'll write prompts underneath Playground.

How to Use ChatGPT to Scrape Any Website

To show you how to utilize ChatGPT to scrape any website, we'll use subslikescript, which lists movies.

Later in this post, I'll show you how to scrape Amazon and Twitter, but let's start easy.

If we try a basic prompt like “scrape the movie titles on this website: https://subslikescript.com/movies” it won’t scrape anything. Here’s when we need some HTML knowledge to build the right prompt for ChatGPT.Let's scrape the above movie titles. First, check the webpage. Right-click any element and choose Inspect.

Then see below.

We identify the element to scrape (those in the red box). The data elements are ul, a, and scripts-list.

Here's the prompt for movie titles.

Scrape this website: https://subslikescript.com/movies with Python and Beautiful Soup. Locate the element with tag “ul” and class “scripts-list”. Scrape all the “a” elements inside. Get the text attribute and print it.

Code generated:

import requests
from bs4 import BeautifulSoup

page = requests.get('https://subslikescript.com/movies/')

soup = BeautifulSoup(page.content, 'html.parser')

scripts_list = soup.find(class_="scripts-list")

all_a_elements = scripts_list.find_all('a')

for element in all_a_elements:
    print(element.get_text())

It extracts movie titles successfully.

Let's scrape Amazon and Twitter.

ChatGPT's Amazon scraping

Consider scraping Amazon for self-help books. First, copy the Amazon link for self-help books.

Here’s the link I got. Location-dependent connection. Use my link to replicate my results.

Now we'll check book titles. Here's our element.

If we want to extract the book titles, we need to use the tag name span, class attribute name and a-size-base-plus a-color-base a-text-normalattribute value.

This time I'll use Selenium. I'll add Selenium-specific commands like wait 5 seconds and generate an XPath.

Scrape this website https://www.amazon.com/s?k=self+help+books&sprefix=self+help+%2Caps%2C158&ref=nb_sb_ss_ts-doa-p_2_10 with Python and Selenium.

Wait 5 seconds and locate all the elements with the following xpath: “span” tag, “class” attribute name, and “a-size-base-plus a-color-base a-text-normal” attribute value. Get the text attribute and print them.

Code generated: (I only had to manually add the path where my chromedriver is located).

from selenium import webdriver
from selenium.webdriver.common.by import By
from time import sleep

#initialize webdriver
driver = webdriver.Chrome('<add path of your chromedriver>')

#navigate to the website
driver.get("https://www.amazon.com/s?k=self+help+books&sprefix=self+help+%2Caps%2C158&ref=nb_sb_ss_ts-doa-p_2_10")

#wait 5 seconds to let the page load
sleep(5)

#locate all the elements with the following xpath
elements = driver.find_elements(By.XPATH, '//span[@class="a-size-base-plus a-color-base a-text-normal"]')

#get the text attribute of each element and print it
for element in elements:
    print(element.text)

#close the webdriver
driver.close()

It pulls Amazon book titles.

Utilizing ChatGPT to scrape Twitter

Say you wish to scrape ChatGPT tweets. Search Twitter for ChatGPT and copy the URL.

Here’s the link I got. We must check every tweet. Here's our element.

To extract a tweet, use the div tag and lang attribute.

Again, Selenium.

Scrape this website: https://twitter.com/search?q=chatgpt&src=typed_query using Python, Selenium and chromedriver.

Maximize the window, wait 15 seconds and locate all the elements that have the following XPath: “div” tag, attribute name “lang”. Print the text inside these elements.

Code generated: (again, I had to add the path where my chromedriver is located)

from selenium import webdriver
import time

driver = webdriver.Chrome("/Users/frankandrade/Downloads/chromedriver")
driver.maximize_window()
driver.get("https://twitter.com/search?q=chatgpt&src=typed_query")
time.sleep(15)

elements = driver.find_elements_by_xpath("//div[@lang]")
for element in elements:
    print(element.text)

driver.quit()

You'll get the first 2 or 3 tweets from a search. To scrape additional tweets, click X times.

Congratulations! You scraped websites without coding by using ChatGPT.

Recent investigations have shown that generative AI can boost hackers and misinformation spreaders.

Since its inception in late November 2022, OpenAI's ChatGPT has entertained and assisted many online users in writing, coding, task automation, and linguistic translation. Given this versatility, it is maybe unsurprising but nonetheless regrettable that fraudsters and mis-, dis-, and malinformation (MDM) spreaders are also considering ChatGPT and related AI models to streamline and improve their operations.

Malign actors may benefit from ChatGPT, according to a WithSecure research. ChatGPT promises to elevate unlawful operations across many attack channels. ChatGPT can automate spear phishing attacks that deceive corporate victims into reading emails from trusted parties. Malware, extortion, and illicit fund transfers can result from such access.

ChatGPT's ability to simulate a desired writing style makes spear phishing emails look more genuine, especially for international actors who don't speak English (or other languages like Spanish and French).

This technique could let Russian, North Korean, and Iranian state-backed hackers conduct more convincing social engineering and election intervention in the US. ChatGPT can also create several campaigns and various phony online personas to promote them, making such attacks successful through volume or variation. Additionally, image-generating AI algorithms and other developing techniques can help these efforts deceive potential victims.

Hackers are discussing using ChatGPT to install malware and steal data, according to a Check Point research. Though ChatGPT's scripts are well-known in the cyber security business, they can assist amateur actors with little technical understanding into the field and possibly develop their hacking and social engineering skills through repeated use.

Additionally, ChatGPT's hacking suggestions may change. As a writer recently indicated, ChatGPT's ability to blend textual and code-based writing might be a game-changer, allowing the injection of innocent content that would subsequently turn out to be a malicious script into targeted systems. These new AI-powered writing- and code-generation abilities allow for unique cyber attacks, regardless of viability.

OpenAI fears ChatGPT usage. OpenAI, Georgetown University's Center for Security and Emerging Technology, and Stanford's Internet Observatory wrote a paper on how AI language models could enhance nation state-backed influence operations. As a last resort, the authors consider polluting the internet with radioactive or misleading data to ensure that AI language models produce outputs that other language models can identify as AI-generated. However, the authors of this paper seem unaware that their "solution" might cause much worse MDM difficulties.

Literally False News

The public argument about ChatGPTs content-generation has focused on originality, bias, and academic honesty, but broader global issues are at stake. ChatGPT can influence public opinion, troll individuals, and interfere in local and national elections by creating and automating enormous amounts of social media material for specified audiences.

ChatGPT's capacity to generate textual and code output is crucial. ChatGPT can write Python scripts for social media bots and give diverse content for repeated posts. The tool's sophistication makes it irrelevant to one's language skills, especially English, when writing MDM propaganda.

I ordered ChatGPT to write a news piece in the style of big US publications declaring that Ukraine is on the verge of defeat in its fight against Russia due to corruption, desertion, and exhaustion in its army. I also gave it a fake reporter's byline and an unidentified NATO source's remark. The outcome appears convincing:

Worse, terrible performers can modify this piece to make it more credible. They can edit the general's name or add facts about current wars. Furthermore, such actors can create many versions of this report in different forms and distribute them separately, boosting its impact.

In this example, ChatGPT produced a news story regarding (fictional) greater moviegoer fatality rates:

Editing this example makes it more plausible. Dr. Jane Smith, the putative author of the medical report, might be replaced with a real-life medical person or a real victim of this supposed medical hazard.

Can deceptive texts be found? Detecting AI text is behind AI advancements. Minor AI-generated text alterations can upset these technologies.

Some OpenAI individuals have proposed covert methods to watermark AI-generated literature to prevent its abuse. AI models would create information that appears normal to humans but would follow a cryptographic formula that would warn other machines that it was AI-made. However, security experts are cautious since manually altering the content interrupts machine and human detection of AI-generated material.

How to Prepare

Cyber security and IT workers can research and use generative AI models to fight spear fishing and extortion. Governments may also launch MDM-defence projects.

In election cycles and global crises, regular people may be the most vulnerable to AI-produced deceit. Until regulation or subsequent technical advances, individuals must recognize exposure to AI-generated fraud, dating scams, other MDM activities.

A three-step verification method of new material in suspicious emails or social media posts can help identify AI content and manipulation. This three-step approach asks about the information's distribution platform (is it reliable? ), author (is the reader familiar with them? ), and plausibility given one's prior knowledge of the topic.

Consider a report by a trusted journalist that makes shocking statements in their typical manner. AI-powered fake news may be released on an unexpected platform, such as a newly created Facebook profile. However, if it links to a known media source, it is more likely to be real.

Though hard and subjective, this verification method may be the only barrier against manipulation for now.

AI language models:

How to Recognize an AI-Generated Article ChatGPT, the popular AI-powered chatbot, can and likely does generate medium.com-style articles.

AI-Generated Text Detectors Fail. Do This. Online tools claim to detect ChatGPT output. Even with superior programming, I tested some of these tools. pub

Why Original Writers Matter Despite AI Language Models Creative writers may never be threatened by AI language models.

Unleash the power of the Option key ⌥

#1 Open a link in the Private tab first.

Previously, if I needed to open a Safari link in a private window, I would:

• copied the URL with the right click command,

• choose File > New Private Window to open a private window, and

• clicked return after pasting the URL.

I've found a more straightforward way.

Right-clicking a link shows this, right?

Hold option (⌥) for:

Click Open Link in New Private Window while holding.

Finished!

#2. Instead of searching for specific characters, try this

You may use unicode for business or school. Most people Google them when they need them.

That is lengthy!

You can type some special characters just by pressing ⌥ and a key.

For instance

• ⌥+2 -> ™ (Trademark)
• ⌥+0 -> ° (Degree)
• ⌥+G -> © (Copyright)
• ⌥+= -> ≠ (Not equal to)
• ⌥+< -> ≤ (Less than or equal to)
• ⌥+> -> ≥ (Greater then or equal to)
• ⌥+/ -> ÷ (Different symbol for division)

#3 Activate Do Not Disturb silently.

Do Not Disturb when sharing my screen is awkward for me (because people may think Im trying to hide some secret notifications).

Here's another method.

Hold ⌥ and click on Time (at the extreme right on the menu-bar).

Now, DND is activated (secretly!). To turn it off, do it again.

Note: This works only for DND focus.

#4. Resize a window starting from its center

Although this is rarely useful, it is still a hidden trick.

When you resize a window, the opposite edge or corner is used as the pivot, right?

However, if you want to resize it with its center as the pivot, hold while doing so.

#5. Yes, Cut-Paste is available on Macs as well (though it is slightly different).

I call it copy-move rather than cut-paste. This is how it works.

Carry it out.

Choose a file (by clicking on it), then copy it (⌘+C).

Go to a new location on your Mac. Do you use ⌘+V to paste it? However, to move it, press ⌘+⌥+V.

This removes the file from its original location and copies it here. And it works exactly like cut-and-paste on Windows.

#6. Instantly expand all folders

Set your Mac's folders to List view.

Assume you have one folder with multiple subfolders, each of which contains multiple files. And you wanted to look at every single file that was over there.

How would you do?

You're used to clicking the ⌄ glyph near the folder and each subfolder to expand them all, right? Instead, hold down ⌥ while clicking ⌄ on the parent folder.

This is what happens next.

Everything expands.

View/Copy a file's path as an added bonus

If you want to see the path of a file in Finder, select it and hold ⌥, and you'll see it at the bottom for a moment.

To copy its path, right-click on the folder and hold down ⌥ to see this

Click on Copy <"folder name"> as Pathname to do it.

#7 "Save As"

I was irritated by the lack of "Save As" in Pages when I first got a Mac (after 15 years of being a Windows guy).

It was necessary for me to save the file as a new file, in a different location, with a different name, or both.

Unfortunately, I couldn't do it on a Mac.

However, I recently discovered that it appears when you hold ⌥ when in the File menu.

Yay!

How to overcome public speaking fear and give a killer presentation

"Public speaking is people's biggest fear, according to studies. Death's second. The average person is better off in the casket than delivering the eulogy."  — American comedian, actor, writer, and producer Jerry Seinfeld

People fear public speaking, according to research. Public speaking can be intimidating.

Most professions require public speaking, whether to 5, 50, 500, or 5,000 people. Your career will require many presentations. In a small meeting, company update, or industry conference.

You can improve your public speaking skills. You can reduce your anxiety, improve your performance, and feel more comfortable speaking in public.

“If I returned to college, I'd focus on writing and public speaking. Effective communication is everything.” — 38th president Gerald R. Ford

You can deliver a great presentation despite your fear of public speaking. There are ways to stay calm while speaking and become a more effective public speaker.

Seven tips to improve your public speaking today. Let's help you overcome your fear (no pun intended).

Know your audience.

"You're not being judged; the audience is." — Entrepreneur, author, and speaker Seth Godin

Understand your audience before speaking publicly. Before preparing a presentation, know your audience. Learn what they care about and find useful.

Your presentation may depend on where you're speaking. A classroom is different from a company meeting.

Determine your audience before developing your main messages. Learn everything about them. Knowing your audience helps you choose the right words, information (thought leadership vs. technical), and motivational message.

2. Be Observant

Observe others' speeches to improve your own. Watching free TED Talks on education, business, science, technology, and creativity can teach you a lot about public speaking.

What worked and what didn't?

• What would you change?

• Their strengths

• How interesting or dull was the topic?

Note their techniques to learn more. Studying the best public speakers will amaze you.

Learn how their stage presence helped them communicate and captivated their audience. Please note their pauses, humor, and pacing.

3. Practice

"A speaker should prepare based on what he wants to learn, not say." — Author, speaker, and pastor Tod Stocker

Practice makes perfect when it comes to public speaking. By repeating your presentation, you can find your comfort zone.

When you've practiced your presentation many times, you'll feel natural and confident giving it. Preparation helps overcome fear and anxiety. Review notes and important messages.

When you know the material well, you can explain it better. Your presentation preparation starts before you go on stage.

Keep a notebook or journal of ideas, quotes, and examples. More content means better audience-targeting.

4. Self-record

Videotape your speeches. Check yourself. Body language, hands, pacing, and vocabulary should be reviewed.

Best public speakers evaluate their performance to improve.

Write down what you did best, what you could improve and what you should stop doing after watching a recording of yourself. Seeing yourself can be unsettling. This is how you improve.

5. Remove text from slides

"Humans can't read and comprehend screen text while listening to a speaker. Therefore, lots of text and long, complete sentences are bad, bad, bad.” —Communications expert Garr Reynolds

Presentation slides shouldn't have too much text. 100-slide presentations bore the audience. Your slides should preview what you'll say to the audience.

Use slides to emphasize your main point visually.

If you add text, use at least 40-point font. Your slides shouldn't require squinting to read. You want people to watch you, not your slides.

6. Body language

"Body language is powerful." We had body language before speech, and 80% of a conversation is read through the body, not the words." — Dancer, writer, and broadcaster Deborah Bull

Nonverbal communication dominates. Our bodies speak louder than words. Don't fidget, rock, lean, or pace.

Relax your body to communicate clearly and without distraction through nonverbal cues. Public speaking anxiety can cause tense body language.

Maintain posture and eye contact. Don’t put your hand in your pockets, cross your arms, or stare at your notes. Make purposeful hand gestures that match what you're saying.

7. Beginning/ending Strong

Beginning and end are memorable. Your presentation must start strong and end strongly. To engage your audience, don't sound robotic.

Begin with a story, stat, or quote. Conclude with a summary of key points. Focus on how you will start and end your speech.

You should memorize your presentation's opening and closing. Memorize something naturally. Excellent presentations start and end strong because people won't remember the middle.

Bringing It All Together

Seven simple yet powerful ways to improve public speaking. Know your audience, study others, prepare and rehearse, record yourself, remove as much text as possible from slides, and start and end strong.

Follow these tips to improve your speaking and audience communication. Prepare, practice, and learn from great speakers to reduce your fear of public speaking.

"Speaking to one person or a thousand is public speaking." — Vocal coach Roger Love

Successful workers don't talk this way.

"I know it's unprofessional, but I can't stop." — Author Sandy Hall

Do you realize your unprofessionalism? Do you care? Self-awareness?

Everyone can improve their unprofessionalism. Some workplace phrases and words shouldn't be said.

People often say out loud what they're thinking. They show insecurity, incompetence, and disrespect.

"Think before you speak," goes the saying.

Some of these phrases are "okay" in certain situations, but you'll lose colleagues' respect if you use them often.

Your word choice. Your tone. Your intentions. They matter.

Choose your words carefully to build work relationships and earn peer respect. You should build positive relationships with coworkers and clients.

These 10 phrases are unprofessional. 

1. That Meeting Really Sucked

Wow! Were you there? You should be responsible if you attended. You can influence every conversation.

Alternatives

Improve the meeting instead of complaining afterward. Make it more meaningful and productive.

2. Not Sure if You Saw My Last Email

Referencing a previous email irritates people. Email follow-up can be difficult. Most people get tons of emails a day, so it may have been buried, forgotten, or low priority.

Alternatives

It's okay to follow up, but be direct, short, and let the recipient "save face"

3. Any Phrase About Sex, Politics, and Religion

Discussing sex, politics, and religion at work is foolish. If you discuss these topics, you could face harassment lawsuits.

Alternatives

Keep quiet about these contentious issues. Don't touch them.

4. I Know What I’m Talking About

Adding this won't persuade others. Research, facts, and topic mastery are key to persuasion. If you're knowledgeable, you don't need to say this.

Alternatives

Please don’t say it at all. Justify your knowledge.

5. Per Our Conversation

This phrase sounds like legal language. You seem to be documenting something legally. Cold, stern, and distant. "As discussed" sounds inauthentic.

Alternatives

It was great talking with you earlier; here's what I said.

6. Curse-Word Phrases

Swearing at work is unprofessional. You never know who's listening, so be careful. A child may be at work or on a Zoom or Teams call. Workplace cursing is unacceptable.

Alternatives

Avoid adult-only words.

7. I Hope This Email Finds You Well

This is a unique way to wish someone well. This phrase isn't as sincere as the traditional one. When you talk about the email, you're impersonal.

Alternatives

Genuinely care for others.

8. I Am Really Stressed

Happy, strong, stress-managing coworkers are valued. Manage your own stress. Exercise, sleep, and eat better.

Alternatives

Everyone has stress, so manage it. Don't talk about your stress.

9. I Have Too Much to Do

You seem incompetent. People think you can't say "no" or have poor time management. If you use this phrase, you're telling others you may need to change careers.

Alternatives

Don't complain about your workload; just manage it.

10. Bad Closing Salutations

"Warmly," "best," "regards," and "warm wishes" are common email closings. This conclusion sounds impersonal. Why use "warmly" for finance's payment status?

Alternatives

Personalize the closing greeting to the message and recipient. Use "see you tomorrow" or "talk soon" as closings.

Bringing It All Together

These 10 phrases are unprofessional at work. That meeting sucked, not sure if you saw my last email, and sex, politics, and religion phrases.

Also, "I know what I'm talking about" and any curse words. Also, avoid phrases like I hope this email finds you well, I'm stressed, and I have too much to do.

Successful workers communicate positively and foster professionalism. Don't waste chances to build strong work relationships by being unprofessional.

“Unprofessionalism damages the business reputation and tarnishes the trust of society.” — Pearl Zhu, an American author

This post is a summary. Read full article here

The best free tools are probably unknown to you.

Modern startups are easy to build.

Start with free tools.

Let’s go.

Web development — Webflow

Code-free HTML, CSS, and JS.

Webflow isn't like Squarespace, Wix, or Shopify.

It's a super-fast no-code tool for professionals to construct complex, highly-responsive websites and landing pages.

Webflow can help you add animations like those on Apple's website to your own site.

I made the jump from WordPress a few years ago and it changed my life.

No damn plugins. No damn errors. No damn updates.

The best, you can get started on Webflow for free.

Data tracking — Airtable

Spreadsheet wings.

Airtable combines spreadsheet flexibility with database power without code.

• Airtable is modern.

• Airtable has modularity.

• Scaling Airtable is simple.

Airtable, one of the most adaptable solutions on this list, is perfect for client data management.

Clients choose customized service packages. Airtable consolidates data so you can automate procedures like invoice management and focus on your strengths.

Airtable connects with so many tools that rarely creates headaches. Airtable scales when you do.

Airtable's flexibility makes it a potential backend database.

Design — Figma

Better, faster, easier user interface design.

Figma rocks!

• It’s fast.

• It's free.

• It's adaptable

First, design in Figma.

Iterate.

Export development assets.

Figma lets you add more team members as your company grows to work on each iteration simultaneously.

Figma is web-based, so you don't need a powerful PC or Mac to start.

Task management — Trello

Unclock jobs.

Tacky and terrifying task management products abound. Trello isn’t.

Those that follow Marie Kondo will appreciate Trello.

• Everything is clean.

• Nothing is complicated.

• Everything has a place.

Compared to other task management solutions, Trello is limited. And that’s good. Too many buttons lead to too many decisions lead to too many hours wasted.

Trello is a must for teamwork.

Domain email — Zoho

Free domain email hosting.

Professional email is essential for startups. People relied on monthly payments for too long. Nope.

Zoho offers 5 free professional emails.

It doesn't have Google's UI, but it works.

VPN — Proton VPN

Fast Swiss VPN protects your data and privacy.

Proton VPN is secure.

• Proton doesn't record any data.

• Proton is based in Switzerland.

Swiss privacy regulation is among the most strict in the world, therefore user data are protected. Switzerland isn't a 14 eye country.

Journalists and activists trust Proton to secure their identities while accessing and sharing information authoritarian governments don't want them to access.

Web host — Netlify

Free fast web hosting.

Netlify is a scalable platform that combines your favorite tools and APIs to develop high-performance sites, stores, and apps through GitHub.

Serverless functions and environment variables preserve API keys.

Netlify's free tier is unmissable.

• 100GB of free monthly bandwidth.

• Free 125k serverless operations per website each month.

Database — MongoDB

Create a fast, scalable database.

MongoDB is for small and large databases. It's a fast and inexpensive database.

• Free for the first million reads.

• Then, for each million reads, you must pay $0.10.

MongoDB's free plan has:

• Encryption from end to end

• Continual authentication

• field-level client-side encryption

If you have a large database, you can easily connect MongoDB to Webflow to bypass CMS limits.

Automation — Zapier

Time-saving tip: automate repetitive chores.

Zapier simplifies life.

Zapier syncs and connects your favorite apps to do impossibly awesome things.

If your online store is connected to Zapier, a customer's purchase can trigger a number of automated actions, such as:

1. The customer is being added to an email chain.

2. Put the information in your Airtable.

3. Send a pre-programmed postcard to the customer.

4. Alexa, set the color of your smart lights to purple.

Zapier scales when you do.

Email & SMS marketing — Omnisend

Email and SMS marketing campaigns.

This is an excellent Mailchimp option for magical emails. Omnisend's processes simplify email automation.

I love the interface's cleanliness.

Omnisend's free tier includes web push notifications.

Send up to:

• 500 emails per month

• 60 maximum SMSs

• 500 Web Push Maximum

Forms and surveys — Tally

Create flexible forms that people enjoy.

Typeform is clean but restricting. Sometimes you need to add many questions. Tally's needed sometimes.

Tally is flexible and cheaper than Typeform.

99% of Tally's features are free and unrestricted, including:

• Unlimited forms

• Countless submissions

• Collect payments

• File upload

Tally lets you examine what individuals contributed to forms before submitting them to see where they get stuck.

Airtable and Zapier connectors automate things further. If you pay, you can apply custom CSS to fit your brand.

See.

Free tools are the greatest.

Let's use them to launch a startup.