Too many files in web application cause long start-up time

Solution Verified - Updated 2 Aug 2024

Environment

Red Hat JBoss Enterprise Application Platform (EAP)
- 6.4.x
- 7.x

Issue

There are too many static contents in web application directory, then recursively scanning cause long time to start server.
The directory which so many files are located must be in web application directory.

Resolution

Check the deployments size and file count and ensure no unexpected file counts

Use `overlay` feature

JBossWeb's <overlay> feature can serve external content libraries and it can avoid recursive scanning. You can workaround the issue by specify outside directory to <overlay> in WEB-INF/jboss-web.xml to add files which are not located inside your deployment.
An example setting of <overlay> in WEB-INF/jboss-web.xml:
```
<jboss-web version="7.0" 
        xmlns:jboss="http://www.jboss.com/xml/ns/javaee" 
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
        xsi:schemaLocation="http://www.jboss.com/xml/ns/javaee jboss-web_7_0.xsd">
    ...
    <overlay>/path/to/contents/</overlay>
    ...
</jboss-web>
```
Note:
- As noted in this knowlege solution, due to This content is not included.BZ#1254062, symbolic links are not followed inside overlay directories before EAP 6.4.7. If you want to use symblic links inside overlay directories, you need to upgrade to EAP 6.4.7 or later.
- <overlay> does not support JSP recompilation
- For EAP 7, it is working from 7.0.8 or later. Please refer this

Use `.index` file

If you cannot set overlay, then you can create .index file for skipping recursive scanning. JBoss VFS can skip the recursive file system visiting if there is a file named root.getPathName() + ".index". The contents is a list of directories under the root (the same with "cd <root>; find . -type d").
Note that the root maybe a jar file or a directory and the .index file is located at the same level with the jar file or the directory (not inside it).

Example:
The deployment test.war looks like below, and we want to skip scanning everything inside the "SkipThis" folder.

test.war
 ├── WEB-INF
 ├── aaa
 │       └── com
 │               └── redhat
 └── SkipThis
           └── xxx
                   └── yyy

The content of test.war.index file will look like below:

SkipThis
SkipThis/xxx
SkipThis/xxx/yyy

Root Cause

JBoss determines the modification timestamp for the deployment, so it can tell if the deployment has been changed and needs re-deploying.
For exploded deployments, it uses the most recent timestamp of any file in the directory, which requires recursively scanning.
There are too many files in web application directory, then recursively scanning cause long time to start server.
The thread dump trace is like below :

"MSC service thread 1-14" prio=10 tid=0x00007f0c1c057000 nid=0x4f72 runnable [0x00007f0c72906000]
   java.lang.Thread.State: RUNNABLE
	at java.io.UnixFileSystem.getBooleanAttributes0(Native Method)
	at java.io.UnixFileSystem.getBooleanAttributes(UnixFileSystem.java:242)
 	at java.io.File.isDirectory(File.java:843)
 	at org.jboss.vfs.spi.RootFileSystem.isDirectory(RootFileSystem.java:107)
 	at org.jboss.vfs.VirtualFile.isDirectory(VirtualFile.java:236)
	 at org.jboss.vfs.VirtualFile.visit(VirtualFile.java:422)
	 at org.jboss.vfs.VirtualFile.visit(VirtualFile.java:423)
	 at org.jboss.vfs.VirtualFile.visit(VirtualFile.java:423)
	 at org.jboss.vfs.VirtualFile.visit(VirtualFile.java:423)
	 at org.jboss.vfs.VirtualFile.visit(VirtualFile.java:423)
	 at org.jboss.vfs.VirtualFile.visit(VirtualFile.java:412)
	 at org.jboss.as.server.deployment.module.VFSResourceLoader.getPaths(VFSResourceLoader.java:252)
	 at org.jboss.modules.ModuleClassLoader.setResourceLoaders(ModuleClassLoader.java:165)
 	at org.jboss.modules.ModuleClassLoader.recalculate(ModuleClassLoader.java:146)
 	at org.jboss.modules.ModuleLoader$3.run(ModuleLoader.java:490)
 	at org.jboss.modules.ModuleLoader$3.run(ModuleLoader.java:484)
	 at java.security.AccessController.doPrivileged(Native Method)
	 at org.jboss.modules.ModuleLoader.defineModule(ModuleLoader.java:484)
	 at org.jboss.modules.ModuleLoader.loadModuleLocal(ModuleLoader.java:381)
	at org.jboss.modules.ModuleLoader.preloadModule(ModuleLoader.java:306)
 	at org.jboss.as.server.moduleservice.ServiceModuleLoader.preloadModule(ServiceModuleLoader.java:148)
 	at org.jboss.modules.ModuleLoader.loadModule(ModuleLoader.java:238)
	 at org.jboss.as.server.moduleservice.ModuleLoadService.start(ModuleLoadService.java:71)

"ServerService Thread Pool -- 92" #213 prio=5 os_prio=0 tid=0x00007ff62cbdd800 nid=0x792d runnable [0x00007ff628462000]
   java.lang.Thread.State: RUNNABLE
	at java.io.FileInputStream.open0(Native Method)
	at java.io.FileInputStream.open(FileInputStream.java:195)
	at java.io.FileInputStream.<init>(FileInputStream.java:138)
	at org.jboss.vfs.spi.RootFileSystem.openInputStream(RootFileSystem.java:51)
	at org.jboss.vfs.VirtualFile.openStream(VirtualFile.java:259)
	at org.jboss.vfs.VirtualJarInputStream.openCurrent(VirtualJarInputStream.java:223)
	at org.jboss.vfs.VirtualJarInputStream.getNextJarEntry(VirtualJarInputStream.java:109)
	at org.jboss.vfs.VirtualJarInputStream.getNextEntry(VirtualJarInputStream.java:80)
	at com.sun.faces.facelets.util.Classpath.searchFromURL(Classpath.java:222)
	at com.sun.faces.facelets.util.Classpath.searchFromURL(Classpath.java:242)
	at com.sun.faces.facelets.util.Classpath.search(Classpath.java:156)
	at com.sun.faces.facelets.util.Classpath.search(Classpath.java:116)
	at com.sun.faces.config.configprovider.MetaInfFaceletTaglibraryConfigProvider.getResources(MetaInfFaceletTaglibraryConfigProvider.java:93)
	at com.sun.faces.config.ConfigManager$URITask.call(ConfigManager.java:1362)
	at com.sun.faces.config.ConfigManager$URITask.call(ConfigManager.java:1331)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at com.sun.faces.config.ConfigManager.getConfigDocuments(ConfigManager.java:742)
	at com.sun.faces.config.ConfigManager.initialize(ConfigManager.java:441)
	at com.sun.faces.config.ConfigureListener.contextInitialized(ConfigureListener.java:227)
	at io.undertow.servlet.core.ApplicationListeners.contextInitialized(ApplicationListeners.java:187)
	at io.undertow.servlet.core.DeploymentManagerImpl.deploy(DeploymentManagerImpl.java:198)

SBR

Product(s)

Red Hat JBoss Enterprise Application Platform

Components

jbossas

Category

Troubleshoot

Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.