I am currently programming with the ultimate goal of developing an information gathering application in my research.
Programming flow(1) Enter a specific word ➤ (2) Set the period ➤ (3) Display the number of searches within the period
Example: ① Enter the word "apple" ➤ ② Select the period (3 items today, yesterday, one week ago) ➤ ③ Number of searches within the period [Apple: ○○○]
I would like to develop this flow of program operation using web scraping (Google) and Tkinter (GUI).
problemCurrently, I am creating programming by dividing it into Tkinter and web scraping.
This time, as a problem on the web scraping side, I would like to show "the number of searches within the period when a specific word is entered".
Whole source codeimport argparse
from time import sleep
from string import ascii_lowercase
from string import digits
import requests
import urllib.parse
class GoogleAutoComplete:
def __init __ (self, test_mode = False, recurse_mode = False):
'hl = ja&output = toolbar&ie = utf-8&oe = utf-8&'\
'client = firefox&q ='
self.test_mode = test_mode
self.recurse_mode = recurse_mode
a = open ('table_hiragana.dat','r')
tango_list = ['a','i','u','e','o','ka','ki','ku','ke','ko','sa','shi' ,'Su','se','so','ta','chi','tsu','te',' and',' na',' ni','nu','ne',' No',' is','hi','fu','he','ho','ma','mi','mu','me','mo','ya','yu' ,'Yo',' et al','ri',' ru','re','ro','wa','wo','n','kya','kyu','kyo' ,'Sha',' Shu',' Sho',' Cha',' Chu',' Cho',' Nya',' Nyu',' Nyo','Hya','Hyu','Hyo ',' Mya',' Myu',' Myo',' Rya',' Ryu',' Ryo',' Gya',' Gyu',' Gyo',' Ja',' Ju' ,'Jo','Bya','Byu','Byo','Pya','Pyu','Pyo']
for tango in a:
tango = tango.rstrip ()
tango_list.append (tango)
if test_mode:
self.chrs = ['A','g', '1']
else: else:
self.chrs = [str (i) for i in tango_list] # Exactly the same as the Perl Japanese syllabary file
def get_suggest (self, query):
buf = requests.get (self.base_url +
urllib.parse.quote_plus (query)). json ()
suggests = [ph for ph in buf [1]]
print ("query: [{0}] ({1})". format (query, len (suggests)))
for ph in suggests:
print ("", ph)
sleep (1)
return suggests
def get_suggest_with_one_char (self, query):
#Suggest word for the keyword itself
ret = self.get_suggest (query)
#Keyword + Suggestion word when blank
ret.extend (self.get_suggest (query +''))
#Keyword + space + suggestion word for 1 character
for ch in self.chrs:
ret.extend (self.get_suggest (query +'' + ch))
return self.get_uniq (ret)
# Exclude duplicates
def get_uniq (self, arr):
uniq_ret = []
for x in arr:
if x not in uniq_ret:
uniq_ret.append (x)
return uniq_ret
if __name__ == "__main__":
phrase = input ('Enter keyword.')
#Google Suggest keyword acquisition
gs = GoogleAutoComplete (recurse_mode = "--recure")
ret = gs.get_suggest_with_one_char (phrase)
#Save to file
fname = "suggest.csv"
with open (fname,'w', encoding ='UTF-8') as fs:
for key in ret:
fs.write (key + "\ n")
Execution result (input word is "apple")
When the command prompt is started
Result ↓
query: [apple] (10)
Apple
Ringo-chan
Apple music festival
Apple recipe
Apple jam
Apple calories
Candy apple
Apple english
Apple cider vinegar
Apple loop
query: [apple] (10)
Ringo-chan
Apple recipe
Apple jam
Apple calories
Apple daughter
Apple english
Apple disease
Ringoro
Apple type
Apple cake
query: [Apple] (10)
ringo apple pie
ringo apple pie calories
ringo apple pie price
ringo apple pie best-by date
Apple Akiei
Apple grip strength
Apple allergy
Apple arrangement
Apple apple pie
Apple allergy baby
query: [Apple] (10)
Apple illustration
Apple dog
No need for an apple doctor
Apple meaning
Weight of one apple
Apple illustration handwritten
One apple calorie
Apple illustration cute
ringo Ikebukuro
Apple illustration
query: [Apple] (10)
Apple rabbit
Apple whiskey
Apple rabbit
Sweetened apples
Sliced apples
Apple Rabbit Real
Apple whiskey
Apple wiki
Other than apple rabbit
Apple rabbit illustration
query: [Apple] (10)
Apple english
Apple nutrition
Apple salt water
Apple picture
How to choose an apple
Apple picture book
Apple ethylene gas
Apple emoji
Apple nutrition composition table
Apple salt water why
query: [Apple] (10)
Apple lunch
Apple weight
Apple sweets
Apple origami
Apple snack
Apple Orin
Apple size
Apple liquor
Apple sweets easy
Apple side dish
query: [Apple] (10)
Apple calories
Apple decoration
With apple skin
Apple kanji
Apple Korean
Apple potassium
Apple decoration rose
Apple cocktail
Apple calorie photo
Apple kanji
query: [Apple] (10)
Apple season
How to cut apples
Apple Kiou
Apple caramelized
Rare varieties of apples
Apple tree
Apple crate
Apple character
Cut and save apples
Apple season word
query: [Apple] (10)
Apple cream cheese
Apple rot
Apple cookies
Apple crumble
Apple comb
Apple cook pad
Apple black vinegar
Apple black
Apple cream cheese snacks
ringo coupon
query: [Apple Ke] (10)
Apple cake
Apple health
Apple blood sugar
Apple blood pressure
Apple cake rice cooker
Apple cake pancake mix
Apple cake frying pan
How to eat apples healthy
Apple chicken
ringo Keio Mall
query: [Apple] (10)
Apple compote
Apple effect
Apple Jonathan
Apple efficacy
Apple compote range
Apple perfume
Apple love sky
Apple compote honey
Apple compote longevity
Apple Kotoku
query: [Apple] (10)
Apple salad
Apple sweet potato
Apple oxidation
Apple sansa
Apple sugar water
Apple sweet potato cake
Apple producing area
Apple Sun Tsugaru
Apple cultivation
Apple San Fuji
query: [Apple] (10)
Apple season
Apple type
Best-by date for apples
Apple harvest time
Apple dietary fiber
Apple cinnamon
Apple Shinano Sweet
Apple consumption
ringo shinjuku
Apple season
query: [Apple] (10)
Apple smoothie
Apple sweets
Grated apples
Apple rice cooker
Apple Spanish
Apple sparkling wine
Apple water content
Soak apples in water
Apple straight juice
Ringo starr
query: [Apple] (10)
The best apple in the world
Apple production
Apple production ranking
Apple making
Apple ingredients
Apple Chiaki
Apple production three-dimensional
From when apples are raw
Apple production world
ringo subconscious
query: [Apple So] (10)
How to grow apples
Apple market
Sauteed apples
Apple sauce
Apple as it is
Apple material
Apple Sonia
From when the apple is as it is
How to grow apples From species
Free shipping on apples
query: [Apple] (10)
Mass consumption of apples
Apple tart
Apple tarte tatin
How to eat an apple
Apple high 2020
Apple carbs
Eat too much apple
Apples expensive why 2020
Apple protein
ringo Tachikawa
query: [Apple Chi] (10)
Ringo-chan
Ringo-chan marriage
Apple Chinese
Apple cheesecake
Apple brown
Apple chocolate
Apple cheese
Long-term storage of apples
Apples don't turn brown
Apple brown
query: [Apple] (10)
Apple mail order
Apple Tsugaru
Grip strength to crush apples
Apple mail order Nagano
Apple mail order Amazon
Apple Tsugaru Features
Apple Tsukurepo
Apple mail order cheap
Apple mail order for home use
Apple ripening
query: [Apple] (10)
ringo store
Apple iron
Apple tempura
Apple regular flight
Apple hand play
Apple natural yeast
Apple fruit picking
Gripping apples
Apple top plate cake
Low temperature cooking of apples
query: [with apples] (10)
Apples and butter
Apple sugar
Apple crested ibis
Apple features
Apple toast
Apple sugar content
Apple Touhou
Order apples
ringo Toyosu
ringo tokyo
query: [Apple] (10)
Apple family
Apple Nakameguro
Apple summer
How many grams of apples
Apple Nagano
Apple pear
Apple name
Long-lasting apple
Apple summer price
Apple whipped cream
query: [to apples] (10)
Boil apples
Crush the apples
Apple carrot smoothie
Apple newton
Apple garlic
Apple pregnant
Apple meat
Apples new Zealand
Apple popular recipe
ringo Nishiazabu
query: [Apple Nu] (10)
Pickled apples
Apple coloring book
How to apply apples
Apple plush toy
Warm apples
Apple cloth
Apple tree
How to apply apples Digital
Apple coloring book illustration
Apple slime
query: [Apple] (10)
Apple price
Apple price is high
Apple cat
Apple necklace
Apple nails
Apple price increase
Apple net
Apple price transition
Apple clay
ringo price
query: [Apple] (10)
Apple tree
Apple compote
Apple time
Types of apples
Apple pound cake
How to store apples
Apple pesticide
Apple forest
Directly from an apple farmer
Apple pesticide
query: [Apple] (10)
Apple flower language
Apple flower
Apple honey
Save half an apple
Apple leaves
Pickled apples in honey
Apple disposal rate
Half an apple calorie
Apple flower
Apple high heel
query: [Apple Hi] (10)
Apple varieties
Apple shelf life
Apple varieties Nagano
Apple varieties time
Apple varieties Tsugaru
One apple weighs
Cool apples
ringo hibiya
ringo Higashiazabu
Apple varieties
query: [Apple Fu] (10)
Apple french
Apple Fuji
Apple free material
Apple fat
Apple filling
Apple hometown tax payment
Apple Fuji season
Apple frying pan
Apple fluffy
Apple felt
query: [to apples] (10)
Apple discoloration
Apple wall
Reason for discoloration of apples
Apple
Apple discoloration time
Apple wall decoration
Apple discoloration sugar
Apple discoloration back
Apple discoloration free study
Apple wall making
query: [Apple Ho] (10)
Apple storage
Apple pancake mix
Apple storage period
Apple pancake mix easy
Apple pancake mix rice cooker
Apple pancake mix pound cake
Apple pancakes
Apple storage room temperature
Apple Hokuto
Apple pancake mix yogurt
query: [Apple] (10)
Apple muffin
Apple round bite
Apples every day
Apple bean knowledge
Whole apple
Apples daily skin
Apples are bad
Apple cartoon
ringo Marunouchi
ringo lineo
query: [Apple] (10)
How to identify apples
Apple honey
Apple mixer
Apple Miki Life
Apple flavor
Apple taste expression
Apple oranges
Apple green
ringo midtown
Miki apple
query: [Apple] (10)
How to peel apples
Apple pesticide-free
Steamed apple bread
Apple mutsu
How to peel apples
Apple swelling
Apple cavities
Steamed apples baby food
Apple mousse
How to peel an apple Rabbit
query: [Apple] (10)
Apple moon
Apple specialty
Apple brand
Apple metaphor
Apple main dish
Apple connoisseur
Apple benefits
Apple menu
ringo menu
Apple mercy
query: [Apples] (10)
Apples
Apple motif
Apple character
Brightness of apples
Apple peach crossing
Apple mold
Apples in the first place
Apple peach
Apple peach allergy
ringo peach
query: [Apples] (10)
Bake apples
Apple baked goods
Apple vegetables
Apple in translation
Apple soft
Roasted apple
Apples lose weight
Apples cheap
Apple yataka
Apple Yamagata
query: [Apple Yu] (10)
Apple dream fortune
Apple bow and arrow
Apples famous prefecture
Apple Yuru Chara
Derived from apples
Apple famous
Apple import
Boil apples
Apple organic acid
Apple Yume Hikari
query: [Apple] (10)
Apple yogurt
Apple sunlight
Apple yogurt cake
Apple night
Apples eat at night
Apple infant food
Apple yogurt recipe
Apple yogurt smoothie
Apple folic acid
How to read apples
query: [Apples] (10)
Apple ranking
Apple Rakuten
Apple Latin
Apple rum
Apple lamp
Apple light
Apple ranking prefectures
Apple wrap
Apple rank
ringo LaLaport Toyosu
query: [Apple] (10)
Apple baby food
Apple dishes
Apple baby food mid-term
Apple baby food late
Apple liqueur
Apple baby food range
Apple baby food recipe
Apple baby food from when
Apple baby food blender
Apple dishes ... omitted
As a result, "apple" and "related words" related to it are given.
In the end, I would like to do the following.
① Select the period (today, yesterday, one week ago)
(2) Extract the number of searches for the word "apple" from Google. (Excluding related words, ex apple + "related words")
I've been searching for hours, but I haven't been able to find the content that gives the number of searches ...
I tried combining it with reference to various blogs, but it became difficult due to malfunctions ... m (__) m
I would like to refer to your answers as much as possible.
-
Answer # 1
Related articles
- python - optuna what is the typical number of searches?
- python - how to put a value in the queue from within the asyncio threadpoolexecutor method
- number of python combinations
- python - how to accept number input even if keypressevent is defined for qlineedit in pyside
- python - yolo i want to detect only a specific class
- python - updating a specific column in each row does not work in the case of duplicate index in dataframe
- python 3x - how to extract row and column numbers with specific values in a dataframe table
- python - valueerror: field'id' expected a number but got'suzukitadashi'
- where is the minor version number of python that starts by default specified?
- i want to aggregate a specific character string by day using the python groupby count function etc
- python - when you want to judge by the number of characters from the back with a regular expression
- python - sumy by specifying the maximum number of characters with sumy
- python - i want to add a line number to the data frame
- substitute the value of a specific line of csv read into python
- a large number of errors when installing the python library
- python - how to list only specific values in a dictionary
- i want to extract specific information from a python string
- python 3x - i want to output the name corresponding to the number entered in the excel file
- python - i want to divide a binary number into two, store it in a list, and represent it in a different decimal number
Excuse me, not the answer, but just in case, scraping to Google is a violation of the rules.
There seems to be an API, so I'll post it for reference (although I haven't used it).
Get Google search results using the Custom Search API