Home>

Twitter API repeats multiple requests to get tweets, but it was a request restriction. Can you tell me what to do to avoid this?

The

code is as follows.

Import JSON
from requests_outhlib import oauth1session
Twitter= OAuth1Session (consumer_key, consumer_secret, access_token_key, access_token_secret)
URL= "https://api.twitter.com/2/tweets/search/all/prod.json"
Enter #Params searchword, number, date, etc.
PARAMS= {'query': '(# ABC)',
        'START_TIME': "2017-10-17T00: 00: 00Z",
        'END_TIME': "2017-10-18T00: 00: 00Z", 'Tweet.Fields': 'CREATED_AT'}
# Get the parameter set above using the GET function from the specified URL
Res= Twitter.get (URL, PARAMS= PARAMS)
IF res.Status_Code== 200:
    # List to process with Pandas later
    CREATED_AT= []
    TEXT= []
    RETWEET_COUNT= []
    favorite_count= []
    # 100 for data more than 100 data
    While True:
        Res= Twitter.get (URL, PARAMS= PARAMS)
        Tweets= json.loads (res.Text)
        Tweet_List= Tweet ["Results"]
        for Tweet in Tweet_List:
            CREATED_AT.APPEND (Tweet ["CREATED_AT"] # Posting date
            Text.append (Tweet ["text"]) # Posted body
            RETWEET_COUNT.Append (Tweet ["Retweet_Count"] # Number of retweets
            favorite_count.append (Tweet ["Favorite_count"] # Like
            User= Tweet ["user"]
        # Target Tweet is 101 or more and there is an NEXT page
        IF "NEXT" in Tweets.keys ():
           Add #next value to parameter
            PARAMS ['Next']= Tweet ["Next"]
            Print (params)
            Tweet_List= Tweet ["Results"]
        If there is no #next page (if it is within 100 or the last page)
        else:
            print ("end page because it is the final page")
            break

The request was limited as follows.

{'TITLE': 'TOO MANY REQUESTS',
 'type': 'about: blank',
 'STATUS': 429,
 'DETAIL': 'TOO MANY REQUESTS'}

It may be caught by the API usage restriction. Can you show me the results of res.json ()?

sei0o2021-06-20 16:52:56

I tried to change my code a little, but still it seems to have been used. When I checked, it is possible to avoid if "wait_on_rate_limit= true" can be avoided, but I do not know where to put it. I'm glad if you know where to put this or another good way.

sashastgy2021-06-20 16:52:56

Wait_On_Rate_Limit is an option of TWEEPY library and seems not to be the function of Twitter API. Please see the answer.

sei0o2021-06-20 16:52:56
  • Answer # 1

    TOO MANY REQUESTSFrom the message, you can see that the TWitter API usage restrictions are caught. If you look at the Rate Limit (restriction restriction), the section is written as follows ("15-minute window" means "in the past 15 minutes").

    App Rate Limit: 300 Requests Per 15-minute Window
    App Rate Limit: 1 Request PER Second

    Use restrictions and how to work around it It is explained in detail in the document. In this case, you can save the search results for a period of time and do not call the API for the same query below. In developing scenes, it is often possible to limit the limit while repeating trial and error, but in that case it will wait.

  • Answer # 2

    TOO MANY REQUESTSFrom the message, you can see that the TWitter API usage restrictions are caught. If you look at the Rate Limit (restriction restriction), the section is written as follows ("15-minute window" means "in the past 15 minutes").

    App Rate Limit: 300 Requests Per 15-minute Window
    App Rate Limit: 1 Request PER Second

    Use restrictions and how to work around it It is explained in detail in the document. In this case, you can save the search results for a period of time and do not call the API for the same query below. In developing scenes, it is often possible to limit the limit while repeating trial and error, but in that case it will wait.