A large number of product image files are stored on the CentOS web server.
What directory structure should I save to display image files as quickly as possible? I'm considering it.
- Save about 1 million image files.
- 10 directories from "0" to "9" are created and nested in 4 layers to create 10 × 10 × 10 × 10 = 10000 storage locations. (Example) /img/1/2/3/4/A.jpg (Pattern A)
- Create 100 directories from "00" to "99" and nest them in two layers to create 100 × 100 = 10000 storage locations. (Example) /img/01/02/B.jpg (Pattern B)
- The frequency of writing (adding/updating) image files is low, and the frequency of reading is high.
For Linux file systems (such as CentOS ext4 and XFS),
Pattern A: Reduce the number of directories and deepen the hierarchy
Pattern B: Increase the number of directories to make the hierarchy shallower
So which will be faster to read?
When the number of image files increases to 100 million in the future, we expect to face performance degradation.
What ideas and improvements are others working on? I asked a question to know.
I would appreciate it if you could give me some hints and reference information. Thank you.
Answer # 1
Save about 1 million image files.
The frequency of writing (adding/updating) image files is low, and the frequency of reading is high.
, I think that storing it in object storage such as Amazon S3 is the quickest solution. You will almost never think for yourself.
Answer # 2
Squid's cache has two levels, and I think the default is about the first level 16 × 2 level 256.
If it is 1 million, it is about 250/directory. It seems that it is still okay if it is 10 million, but it is a number that thinks that it is 100 million.
If you have to assume 100 million (the number of requests will naturally increase), you need to consider whether it can withstand a single server. Relying on external services is an option, but it doesn't mean you don't have to think about anything. Since the risk of making it yourself is not lost, it is just converted to billing, so you need to consider it carefully.
- Linux file directory structure (newbie version)
- linux grep "no such file or directory" message meaningless
- Detailed explanation of Linux jdk installation directory, Linux uninstall jdk, rpm command, rm command parameters
- vue-cli directory structure explained in detail
- Tomcat directory structure in detail
- How to find the corresponding process name and directory by the PID number in Linux
- 10 ways to delete files in a directory in Linux
- Vuex project structure directory and some simple configuration introduction
- Python implement tree print directory structure
- Linux decompression method to specified directory
- Extract all directory and file names in a folder in bulk in Linux
- The structure and purpose of each folder under Linux (super detailed)
- On Linux file directory introduction and file color difference
- Interpret the file directory structure and description generated by vue
- Detailed vue file directory structure
- Directory structure and description of the project created based on vue-cli
- Talking about Storm's directory structure on zookeeper
- The clever use of the Linux Shell to find files in a directory using a for loop
- Find a running nginx directory in a Linux system
- i want to exclude the shell script linux command git directory from the search target -i don't know how to use prune
- python 3x - typeerror: 'method' object is not subscriptable
- python - you may need to restart the kernel to use updated packages error
- xcode - pod install [!] no `podfile 'found in the project directory
- android studio - emulator: dsound: could not initialize about the error message directsoundcapture
- android studio - unresolved reference comes out in kotlin
- vuejs - [vuetify] unable to locate target [data-app] i want to unit test to avoid warning
- django - oserror: [winerror 123] the file name, directory name, or volume label syntax is incorrect : '<frozen importlib_boot
- mysql startup failed [error] innodb: the innodb_system data file 'ibdata1' must be writable
- python 3x - import error "cannot import name '_imaging'"