Web Hosting Java, JSP, Tomcat 6, J2EE, Servlets, Struts, Jboss

Scalability Basics To understand software performance and scalability

Scalability Basics To understand software performance and scalability issues on a multiprocessor, we have to have a rudimentary grasp of the underlying architecture. Since multiprocessors evolved out of uniprocessors, let’s step back and start with a quick overview of the uniprocessor computer architecture. The mainstream uniprocessor architecture consists of a single CPU and a single main memory module. Program data and instructions reside in main memory. Since CPU speed is at least an order of magnitude faster than memory, we also have an additional (very) fast memory inserted between the processor (= CPU) and main memory. This fast memory is also known as the cache. The cache provides faster access to program data and instructions. The cache may be split into two physical units, one for data and another for instructions. We’ll ignore that distinction and refer to it as a single logical entity. The processor needs at least one memory access per instruction in order to retrieve the instruction itself. On most instructions the processor may need additional memory references for data. When the processor needs memory access, it looks in the cache first. Since cache hit-ratios are upwards of 90 percent, slow trips to main memory are infrequent. This whole story is summarized in Figure 7.1. Figure 7.1. The Uniprocessor architecture An application is a single process or a set of cooperating processes. Each process consists of one or more threads. The scheduling entity on modern operating systems is a thread. The operating system does not execute a process, it executes a thread. Threads that are ready to execute are placed on the system’s run queue. The thread at the front of the queue is next in line for execution (Figure 7.2). Figure 7.2. Threads are the scheduling entities Page 146

Hint: This post is supported by Gama hrvatski web hosting services

Scalability Basics To understand software performance and scalability

Page 147

Hint: This post is supported by Gama hrvatski web hosting services

Chapter 7. Scalability and Multithreading As a server-side

Page 145

Hint: This post is supported by Gama besplatan domen provider

Chapter 7. Scalability and Multithreading As a server-side

Chapter 7. Scalability and Multithreading As a server-side programmer, my picture of the world is somewhat distorted by my experience. I have never worked on software that was single-threaded. Multithreaded design and implementation is the backbone of server-side programming, but they are definitely not limited to the server alone. Multithreading comes in handy on the client side as well. Why would you want to multithread your software? Three good reasons immediately come to mind: . Increased throughput . Reduced response time . Better CPU utilization Take a Web server, for example. Practically all of the dominant Web servers are multithreaded. Each incoming HTTP request is assigned to a server thread that will “escort” the request from its arrival until its departure in the form of a response. That thread could easily get blocked waiting on a page-fault, disk I/O (while reading a file), or socket I/O (waiting to receive data from a CGI script or a database.) If a Web-server thread gets blocked, there are usually plenty of other threads that can step up and continue serving other requests and keep the server cruising at full speed. This is where increased throughput comes into play on the server side. Multithreading also improves CPU utilization because there’s usually some thread ready to make productive use of the CPU. As mentioned earlier, multithreading is not limited to the server side; it also plays a major role on the client side. When you want to bail out of a very slow browser download, you hit the cancel button and the download aborts. If the browser is single-threaded, it isn’t very responsive. That single thread becomes tied up with the document download and ignores your plea to abort. Multithreaded programming is what makes the clients appear so responsive. Multithreaded programming has a special role in Java. When it comes to raw speed, Java programs are slower than C and C++ programs, and that is not going to change any time soon. In theory, you would expect this to be a show-stopper on the server side. In practice, however, Java is now carving itself a significant territory as a viable server-side programming language. This is not just because Java is so portable and its programmers so productive. It is also not because customers all of a sudden abandoned their obsession with performance. It is, to a large extent, because hardware is cheap and you could always throw in a 12-way SMP monster to elevate Java application performance to the required level. On the other hand, a powerful SMP machine does not guarantee improved performance. A serious programming challenge is still facing the Java developers: to design and implement our software in a scalable fashion. Faced with the task of speeding up a Java application, you generally have the following options: . Tune your application. Reduce the application’s pathlength by optimizing the code. This approach has been the focus of our discussion up to this point: speeding up a single thread of execution regardless of the underlying machine architecture (UP or SMP.) . Upgrade processing speed. A faster CPU should result in faster execution for CPU-bound workloads. . Add processors. A multiprocessor machine consists of multiple CPUs. In theory, multiple CPUs should outperform a single CPU of identical speed. This is what scalability is all about. The challenge in scalability for an application code is to keep up with the additional processing Page 144

Hint: This post is supported by Gama besplatan domen provider

Optimization 38: Last In, First Out On multithreaded

Key Points . Object creation could be expensive. It includes the cost of memory allocation and initialization of the object’s direct instance variables as well as those instance variables inherited from each and every superclass. . Object destruction is performed implicitly as part of garbage collection. Like creation, object destruction is not free. It contributes to the overall cost of garbage collection. . Many class implementations provide a reset operation to restore an object to its initial state. Often, this is much more economical than polluting the code with new objects. It also lightens the load on garbage collection. . Recycling a single object is a special case of an object pool of size 1. It is the simplest and fastest recycling mechanism. Otherwise, if you cannot tell in advance how many objects are required during execution, you can still tap into the benefits of recycling by using object pools. . In the special case of a thread pool, it is more efficient to enforce a LIFO policy to better utilize the system’s limited cache and memory space. Page 143

Hint: If you are looking for very good and affordable webspace to host and run your j2ee hosting application check Virtualwebstudio j2ee web hosting services

Optimization 38: Last In, First Out On multithreaded

Optimization 38: Last In, First Out On multithreaded servers you will often encounter a performance requirement to pool server threads. Threads are expensive to create and destroy and therefore are ideally suited for object pooling. This is, however, a special pool with additional performance nuances. A Java thread can easily drag with it one MB of memory dedicated to required JVM data structures, as well as additional user-defined data. The last thread to hold the CPU has already gone through the work of loading its pages onto memory and a significant portion of its data and code segments onto the instruction and data cache. It would be foolish to throw all this work away. The thread pool should dole out threads in a last in, first out (LIFO) order. That way, we can have lots of threads in the pool (for a rainy day) but only a few that are truly active and using the CPU. This approach makes a much more effective use of the limited cache and memory space. A smaller number of active threads may fit all of their data and code into the cache. A LIFO thread pool keeps the cache and memory “warm.” Otherwise, if the pool gives equal opportunity to all its threads, we are likely to have a “cold start” for each thread from a caching and paging perspective. Some containers, such as a Stack and Vector, automatically lend themselves to the LIFO policy. The Stack won’t let you do it any other way, and the Vector should have elements added and removed from the back only (see Optimization 17). In a linked list, on the other hand, you could erroneously attach your threads to the end of the pool and remove from the front. That would be a costly caching and paging mistake. Page 142

Hint: If you are looking for very good and affordable webspace to host and run your j2ee hosting application check Virtualwebstudio j2ee web hosting services

Page 140

Page 141
Note: If you are looking for cheap and quality provider to host and run your java application check Astra java hosting services

Page 140

Page 140
Note: If you are looking for cheap and quality provider to host and run your java application check Astra java hosting services

Page 138

Optimization 37: Object Pools The previous optimization was made simple by the fact that we knew in advance that we needed only a single Vector object in the course of the computation. If you know in advance that you need only one instance of a resource that is used repeatedly, you could recycle and reuse that same instance over and over. This is actually a degenerate case in which the pool size is 1. In the more general scenario, you don’t necessarily have advance knowledge of how many instances are required. It could be 0, 1, or many. In that case, you may have to acquire the resources on the fly but you still want to avoid the cost of creating and destroying expensive resources on a regular basis. The answer here is pooling. After a brief transient state of populating the pool, acquiring and releasing the pooled resource is significantly cheaper because acquire and release operations do not really equate to expensive create and destroy. We merely put and get to and from the pool. Resources that often get pooled are threads, JDBC connections, persistent sockets, as well as other user-defined heavyweight objects. The following code is a rudimentary object pool: public class ObjectPool { protected Object objects[]; protected Class cls; protected int head; public ObjectPool(String className, int size) throws ClassNotFoundException, IllegalArgumentException { this(Class.forName(className), size); } private ObjectPool(Class p_class, int size) throws IllegalArgumentException { if((null == p_class) || (size <= 0)) { throw new IllegalArgumentException("Invalid arguments"); } head = 0; cls = p_class; objects = new Object[size]; } public Object getObjectFromPool() throws InstantiationException, IllegalAccessException { Object obj = null; synchronized(this) { // If there is an available object in // the pool, then return it. Else, create // a new instance of the class and return it. if (head > 0) { head–; obj = objects[head]; objects[head] = null; } else { obj = (Object)cls.newInstance(); } Page 139

Hint: If you are looking for very good and affordable webspace to host and run your java hosting application check Virtualwebstudio java web hosting provider

Page 138

Page 138

Hint: If you are looking for very good and affordable webspace to host and run your java hosting application check Virtualwebstudio java web hosting provider