College】 【weak intrusion detection system test and evaluation (2).

4 test and evaluation of the performance indicator IDS
In our analysis of the IDS of the performance, the main consideration detection system effectiveness, efficiency and availability. Validation of detection mechanism detection accuracy and credibility of the system test results, it is to develop the design and use IDS premise and aims to test the key indicators to assess the IDS, and efficiencies are handled from the detection mechanism of data, and economic considerations, also is to focus on the detection mechanism improved price-performance. Availability including system scalability, availability of user interface, ease of deployment configuration. Validation is to develop the design and use of the premise and IDS, thus the purpose is to test the key indicators to assess the IDS, efficiency and availability on IDS performance also plays a major role. Efficiency and availability of the various aspects of the system design. This section from the detection of effectiveness, efficiency and availability perspective on testing and evaluation of the performance indicator IDS implementation analysis of the discussions.
4.1 detection rate, false alarm rate and detection reliability
Detection rate refers to the monitoring system to be intrusion attacks, detection system can correct alarm.　　False alarm rate refers to the inspection system detects when the probability of a false alarm. Testing the credibility of the detection system detects the results of credibility, this is a test to assess the IDS of the most important indicators. Actual IDS implementations are always in the detection rate and false alarm rate hovering between the detection rate high for the false alarm rates will improve; equally false alarm rate decreases, the detection rate will be reduced. In General, IDS products will both take a compromise and be able to perform the adjustment, in order to adapt to different network environments. United States of Lincoln Lab sink characteristics(ROC, ReceiverOperatingCharacteristic) curve to describe the performance of IDS. The curve accurately depicting the IDS detection rate and false alarm rate changes. ROC is widely used in the input does not determine the evaluation of the system. According to an IDS on different criteria (allowed within the threshold of changes, such as anomaly detection system parameters such as the alarm threshold) false alarm rate and detection rates, the false alarm rate and detection rates as abscissa and ordinate, you can make the IDS corresponding to the ROC curve. ROC curves and IDS detection threshold has a corresponding link.
In the assessment of the specific implementation process IDS in addition to the IDS of the detection rate and false alarm rate, often also considered alone and these two indicators closely-related factors, such as to detect intrusion feature number, IP fragment reassembly TCP stream reassembly ability and capacity. Obviously, the number of intrusion detection feature, the detection rate is also higher. In addition, because the attacker in order to increase the difficulty of detecting even bypass detection of IDS, often sends some special design group. In order to increase the detection rate of IDS IDS of false alarm rate, often need to take some IDS corresponding methods, such as IP fragmentation and TCP stream reassembly. Because of a single data packet can cause a lot of false positives and false negatives, so IP fragment reassembly can improve the detection accuracy. IP fragment reassembly of metrics have three character parameters: reorganization of the maximum number of IP fragments; at the same time the number of IP packet; can do the reorganization of the largest IP data packet length, TCP stream reassembly for dialogue on the complete network, which is the implementation of network IDS to perform an analysis of the application layer. Such as checking the message contents. Attachment, check the FTP transmission of data, prohibit access to harmful sites, determine illegal HTTP requests, etc. These two capabilities will directly affect the reliability of detection IDS.
4.2IDS itself against attack capability
And other systems, IDS are often security vulnerabilities. If the IDS attack succeeds, the direct cause of the alarm malfunction, an intruder in subsequent conduct will not be logged. Therefore IDS you must first be their own security. IDS itself hack-capability that is reliability of IDS to measure IDS for those specially designed directly to IDS for the target of the attack resistance. It is mainly embodied in two ways: first, the program itself in a variety of network environments to work properly; the second is that communication between the various modules can not be destroyed, not fake. In addition to special consideration to defend against denial of service attack. If IDS itself does not run correctly, you also lost its significance of defence. But if the system of communication between the various modules is compromised the system alarm, detection result is questionable, should have a good communication mechanism for ensuring the security of communication between modules and the queries to speed recovery.
4.3 other performance indicators
Delay time. Measuring latency refers to the attacks to IDS intrusion detected the delay time between. Delay time directly linked to the degree of intrusion attacks.
Resource consumption. The system to achieve a detection effectiveness on resource needs. Typically, the validity of the same detection, resource requirements, the lower, the better the IDS of the properties, the ability to detect the intrusion.
Load capacity. IDS have the load carrying capacity of its design, the load carrying capacity is exceeded, the performance will be varying degrees of decline. For example, in normal circumstances IDS detects an attack but load large may not detect the attack. Investigation of load carrying capacity of the detection system is the observationof different size of network traffic, CPU memory of different intensity and other system resources using key indicators on the IDS (such as the detection rate and false alarm rate).
Log, reported good, reporting, and response capabilities. Log capacity refers to the measurement system's ability to save the log, in accordance with the specific requirements of the capacity of selected log contents. Alarm capability refers to the intrusion detected, to all of the pieces, people, the ability to send the alarm signal as well as additional information in the alert. Report capability refers to produce the report, provide intrusion report, building a query and save the report. Responsiveness is detected inPenetration ability after further processing, including blocking intrusions, tracing intruders, records, intrusion evidence, etc.
The availability of the system. Mainly refers to the system installation, configuration, management and use of convenience and user-friendliness of the system interface, attack code library maintained by the simple level, etc.
In short, the IDS is a more complex system, perform a test on IDS and assessments and IDS itself, and application of environment-related IDS. Testing process involves operating environment, network environment, tools, software, hardware, etc. We need to consider the effect of intrusion detection, to consider the use of the system after it on the actual system, sometimes we have to compromise to consider these two factors.
5 to perform a test to assess a IDS used by the data
To perform a test to assess the IDS, also is to enable IDS to access to defend the system data execution detection, to determine whether the discovered one of detected system intrusion. IDS you want to test assessment, and of course the most accurate data is based on the actual operating environment of the data, but this is usually not feasible. Because agencies data contains some private information, they do not want to expose these data, and even with institutions willing to open their own data, may not be suitable for general testing, because the data for a specific sector with significant unique characteristics, has certain limitations, nor good repeatability. To this end, specific tests, most with some test tools. Through these tools to generate the IDS of test data.
Test evaluation data generated to meet the following multiple conditions, that is, the data must be generated automatically, do not require human intervention; to have a certain degree of reproducibility, that is required to produce the same data; to have a certain degree of robustness, in the monitoring of the condition, you can run a long time.
Test IDS data evaluation consists of two parts, part of the training data, and another part is the actual test data. This two-part data include normal data and intrusion data. Only in the context of normal data, on the results of the evaluation of testing IDS is objective and thorough. Intrusion in the background of the data under the cover, is the chance of detection system that is significantly reduced. While the IDS may also be a normal flow behavior misjudgment of the attack, produce a false alarm. Training data is used to help establish normal behavior of IDS models, adjust the configuration of the parameter IDS.　　In the training data, intrusion data are clearly labeled. Test data used to perform a test on the test system, including intrusion data does not identify. Usually use the following three main points generated contains both normal communication data and attacks to public data: grab a normal situation and be controlled attack run communication data. Due to privacy and security questions obviously unworkable; from actual operating data clean up secret information. And to join the attack, and this unfeasible, as it is difficult to clean up the secret information; in an internal network renewal normal communication and attack data, we use the essentials.
Reconstruction of normal communication and attack data also is simulation user operations, simulation of intrusion. Simulation of a user action that is generating the user a variety of normal use patterns, these models help IDS based on anomaly detection of establishing normal behavior of the model, and to users the normal mode data as background for detecting intrusion, for sure the communication data IDS normal run-time detection rate and false alarm rate is very necessary. Simulated intrusion should as much as possible to cover a variety of types, new attack only appeared in the test data. Design of the attack to take into account a number of questions. To analysis attack mechanisms and, in the test system tests for ease of analysis and adjustment. Analysis to determine if an attack in the test environment can work, can require new software or services support. Design of novel attacks to find that is not used by the system or network vulnerability. Following on the user's normal mode of simulation and intrusion simulation performed separately.
At present, most of the following three main points to simulation network user behavior, namely general session generation tool, test, package and recording replay actual data. General session key generated tools based on finite automata to generate user all possible actions. Each operation has a certain procedures, such as FTP operations, first of all it to complete the three steps of the TCP handshake is the original of the connection, and then to enter a user name and password, the user name password through after reading the content on the FTP server and download or upload, all operations are complete, leave the server over a TCP session. According to the common regulations, you can generate a generic session that simulate user actions. However, this idea is applicable only to test the limited set of commands, FTP clients, for example to simulation, but cannot shell client, and the simulation of the simulation there is some doubt, because the user actions and server-side response are uncertain, emulation does not completely simulate user actions. Operating system developers with test package is more basic analog essentials, is often used to test to assess the performance of the operating system services and application service software can not be achieved by the design specification. However, this test does not give the users what kind of action can only tell our system on request. Recording replay essentials is record the user's normal activity data, and then test the platform activities of replay user processes. The essentials required user activity records to enough.
User simulation of normal behavior including network traffic simulation, simulation of normal use of the host. The majority of network IDS, or the network IDS of the majority of work at the network layer or above the network layer, data on the Web group depending on the protocols implementing the appropriate analysis. Therefore, in a simulation of network traffic, to simulation in a wide range of protocols in the application's flow. Typically, the actual flow analysis, statistical calculations, the individual agreements by time's flow probability distribution is used as a model for the simulation of various protocols, respectively.
The application of the host can be divided into two parts: the host network services provided by the application and the host application, that is, the user directly to execute commands on the host. The corresponding host normal use of simulation to be divided into two parts, i.e., host Web Services application simulating normalTrue and host directly use of simulation. On the host-provided network services to perform the normal use of simulation, you can use two main points. One is the traverse method, that is, identify a service allows use of all normal mode, then by the emulator, follow these models in order to access the service implementation. The second is the actual sampling method to obtain the real network environments for a service actually use data analysis application mode appears, then, based on the results of the analysis performed simulation established simulation model. The essentials and network traffic simulation points. Both of these points has advantages and disadvantages, simulation, simulation services according to the situation. Because the user's behavior was different, the nature of the work will vary considerably, so the host should direct the application of simulation will be divided into different types of users (such as administrators and ordinary users), according to the different user types to write different scripts, for host directly use of simulation. Since different users use habits changed much, and even if the same user using habits also comes with a lot of randomness, which makes it more difficult to significantly add simulation. Testing and evaluation in actual IDS, generally only the normal use of the simulation hosts a representative subset.
Attack simulation is the core of the evaluation environment, but also to perform a test of key IDS. Attack simulation to collect as many different types of attack. The number of various attacks too large, it is not possible on all attacks are performed simulation. Reference software testing field equivalent partitioning Essentials (equivalencepartitioning), to perform the attack simulation, the general classification of the attack, and then select each category in a typical attack points perform simulation test. After selecting the type of attack, the simulation is performed according to the intruder attack simulation of steps. When you construct the attacks but also keep an eye on new data. Attack forms secret attacks, side-by-side execution of the attack. Relative to old-style attack, attack form obvious attacks and serial execution of the attack, the attack forms on measurement results of impact may be greater.
At present, the test data in the format in which most of the Tcpdump data format and data formats, as BSM Windows system is widely used, WindowsNT log format has gradually taken into account. In the test data, MIT Lincoln Laboratory data comparison is complete, it includes certain time of the training data and for the last actual test instrumentation data. For network traffic simulation tools have Anzen Corporation nidsbench and University of California at development of intrusion detection testing platform. Tcpreplay and fraqrouter nidsbench includes two parts. Tcpreplay is the tcpdump group of replicated data, restore network replay of actual operating status; fraqrouter features is by constructing a series from IDs detection of attacks to test the accuracy of detection systems and security. The University of California IDS software testing platform using Tcl-DP (TooLCommandLanguageDistributedProgramming) tool development. It contains a total of four groups: basic session command set, the synchronization command sets, communications command set, record, replay command set. These commands are used for simulation of the intruder's basic operation, according to specified requirements to produce the event, the implementation of concurrent processes of communication, as well as record user session operation command sequence and then replay them. In addition, MIT Lincoln Laboratory have developed a non-real-time IDS performance assessment tool that dynamic replay large amounts of data.