Search for a character string that has [Name, Furigana, Score] as a set and search for a character string that matches the name or furigana.
I want to speed up the process as much as possible. (It takes time to search because there are tens of thousands of directory data)
What I did
①Store the name list in the 2D array of the list and search from the beginning → It took time
② Search the database by executing the SQL statement every time after putting the list in the database → It is faster than ① but still takes time
Because I don't have much knowledge about python, I'd like to know what speedups can be considered. (You can use this library, do this, etc.)
Thank you m (_ _) m
for name in names: if name == meibo ["name"] or name == meibo ["furigana"]: print (meibo ["score"])
Answer # 1
Whether it should be sorted and binary search, or put it all in a hash table (dictionary). Basically, the latter should be faster.
However, whether it's a sorted array search or a hash table, it seems that it will take some time to make it first (although some tens of thousands may have been seen), I made it Dump things with pickle, etc., and make a program that reads them afterwards.
Answer # 2
When tuning, first measure where the time is spent in the profiler (cProfile).
Reference: Python profiler
If SQL, paste
phoneticof the table, and set the
WHEREcondition to 1 Whether it should be in the form to acquire the matter.
c.execute ('SELECT * FROM meibo WHERE name =? OR furigana =?', (name, token.base_form,))
- python 3x - there is a part of the python code that i can't understand
- [python] i want to determine if there is an element in the list with an if statement
- python - selenium: there is no element in the page that can be used as a method to operate the login button click
- python - is there a condition judgment that doesn't use elif or else?
- python - please tell me how to find the cumulative sum at high speed
- mass random number generation speed in python
- python 3x - there are some things that can not be implemented in python3, so i would like advice if possible
- how to use numba to speed up python library ode
- python - i want to speed up the element extraction of a two-dimensional array in numpy or pytorch
- i want to speed up this program with python
- python - difference in execution speed (program executed at a specified time)
- python - is there a more concise way to print from monday to sunday?
- python 3x - there are some expressions i don't understand in python code
- python - atcoder abc171 f problem how to speed up
- python 3x - please let me know because there is a part of the code that i do not understand in the if statement
- python - how to remove a list from a tuple and store only the value
- http error 503: service unavailable when doing amazon scraping with python
- python - i want to simplify the program that sequentially retrieves from the list
- python - regarding the problem that tries to decompose a string apostrophe using a dictionary but fails
- python - common behavior for everything in django
- python - i want to split one array into two without duplication
- about the problem that float subtraction does not work in numerical calculation in python
- python - something went wrong in the jupiter lab
- python - how to deal with error when converting pandas datetime type
- i want to read python time series data while shifting by 1 day