Experimental and industrial validation of design patterns

By Poliakova Ksenia (kapoliakova@edu.hse.ru)

Design Patterns are tested within quite long period of time and also they are proven solutions to common problems encountered in software development. They are a kind of “building blocks” for creating flexible, scalable and maintainable systems. However, before a design pattern is considered to be useful and applicable in practice, it must undergo rigorous validation. Validation, in this context, is the process of assessing the pattern’s compliance with its stated goals and its suitability for use in real-world conditions. The talk will cover in detail two key aspects of design pattern validation: experimental and industrial validation.

Design patterns are reusable solutions to common design problems. They describe how to properly organize code so that it becomes more flexible, modular, and scalable. Some examples of the most popular design patterns: Singleton, Factory, Observer, Strategy, etc. [2].

Validation Goals:

1. Effectiveness testing: ensuring that the pattern solves the problem. First and foremost, validation aims to rigorously test the effectiveness of the pattern. This means that it is necessary to carefully evaluate how well the pattern solves the specific problem for which it was developed.
2. Applicability evaluation: identifying the optimal context of use. Design patterns are not universal solutions that are suitable for all cases. It is important to accurately determine the context in which the pattern will be most effective and appropriate. This means examining the specifics of different types of projects, systems, and architectures, as well as analyzing the requirements placed on them.
3. Identifying limitations: finding potential problems and risks. No pattern is perfect, and each has its own shortcomings, problems, and potential risks associated with its application. It is necessary to identify these weaknesses so that developers can anticipate them in advance and avoid mistakes when using the pattern.
4. Versatility verification: assessing the applicability in various conditions. It is necessary to investigate to what extent the pattern can be adapted and applied in different projects, technologies and development environments. This involves checking how easy it is to integrate the template with existing tools and libraries, and how compatible it is with different platforms and programming languages.
5. Template improvement: a continuous improvement process. Validation is not just a check, but a feedback loop that allows you to identify shortcomings and make changes to the template to improve its efficiency, reliability, and applicability. Based on the identified problems, changes can be made to the template description, recommendations for its use can be added, or even a complete redesign of the template architecture.

Experimental validation is the process of evaluating design patterns under controlled conditions using specially designed experiments. The main goal is to obtain quantitative and qualitative data on the effectiveness of the template, its performance, flexibility and other important characteristics [3].

Goals of experimental validation

1. Performance measurement. Evaluation of the pattern's impact on system performance (execution speed, memory usage, etc.).
2. Scalability assessment. Checking how the pattern copes with increasing load and data volume.
3. Flexibility analysis. Evaluation of how easy it is to change and extend the system using the pattern.
4. Testability assessment. Checking how convenient and simple it is to test the code using the pattern [3].

Experimental validation methods

1. Micro-benchmarks. Conducting tests to measure the performance of specific parts of the code using the pattern.
2. Stress testing. Evaluation of how the system copes with high loads and extreme values.
3. Unit testing. Creating a set of unit tests to check the correct operation of the code using the pattern.
4. Experiments with changing parameters. Evaluation of the effect of various parameters on the efficiency of using the pattern.

Experimental validation begins with a clear definition of the experimental objectives, defining what exactly needs to be tested. Then test rig is created, including the necessary hardware, software, and test data, forming a controlled environment for conducting experiments. After that test scenarios are developed, defining a set of tests designed to validate the template. During the experimentation phase, these tests are executed and the resulting data is carefully collected. Finally, an analysis of the results is carried out, including processing and interpretation of the collected data, as well as comparison with the results of a control group (if any) to obtain objective conclusions about the functioning and effectiveness of the tested template.

Industrial validation is the process of evaluating design patterns in real conditions, on real projects, and with the participation of real developers. It allows you to evaluate the applicability of the pattern in practice and its impact on the efficiency of team development.

Goals of industrial validation:

1. Ease of use assessment. Determine how easy it is for developers to understand and apply the pattern in practice.
2. Impact assessment on teamwork. Check how the pattern affects communication and collaboration between developers.
3. Maintainability assessment. Determine how easy it is to maintain and evolve the code that uses the pattern in the long term.
4. Project cost impact assessment. Analyze the pattern's impact on development time, number of errors, and overall project cost.
5. Scalability assessment in real projects. Analyze the pattern's behavior in large and complex systems.

Industrial validation methods.

1. Developer survey. Collect feedback and opinions from developers using the pattern in real projects.
2. Code analysis. Assess the structure, complexity, and readability of the code that uses the pattern.
3. Code review. Reviewing code by other developers to ensure that the pattern is used correctly.
4. Project metrics analysis. Measuring development time, number of errors, code complexity, and other metrics related to pattern application.
5. Case study. Studying specific projects where the pattern was applied and analyzing their results.

Industrial validation process In the context of the study of the implementation of a development pattern, the key stage is the selection of a real project on which testing and integration will be carried out [5]. After the project is selected, the pattern is directly implemented, accompanied by training developers in its use to ensure effective application. This is followed by data collection, including observation of the development process, receiving feedback from developers and measuring key project metrics, which allows us to assess the impact of the pattern on work processes. The collected data is analyzed and interpreted to identify patterns and trends, after which conclusions are drawn about the applicability of the pattern in real conditions, as well as its advantages and disadvantages identified during practical application.

Validation of design patterns is an integral part of the software development process to ensure their effectiveness and applicability. Experimental validation provides quantitative data on the effectiveness of a pattern under controlled conditions, while industrial validation evaluates its applicability in real projects [3]. Both types of validation are important to form a complete and comprehensive understanding of the advantages, disadvantages, and limitations of using design patterns [1]. A combination of experimental and industrial validation is the most effective approach to verifying design patterns. The results of validation help developers choose the most appropriate patterns to solve specific problems and improve the quality of software.

Experimental and industrial validation of design patterns are two complementary processes that provide a comprehensive assessment of their effectiveness and applicability. Experimental validation provides quantitative data under controlled conditions, while industrial validation provides an understanding of the real context and impact of the pattern on software development. Taken together, these methods allow choosing the best solutions for building reliable, flexible and scalable systems, which is an important factor in modern development.

1. Nadolsky V.V. Calculation and design of a flange connection of rectangular cross-section elements subject to central tension - 2018. - No. 16. - P. 121-130.

2. Tur V.V., Nadolsky V.V. Concept of designing building structures based on numerical resistance models - 2022. - No. 6 (104) - P. 78-90.

3. Perelmuter A.V., Tur V.V. Are We Ready to Move to Nonlinear Analysis in Design? - 2017. - Vol. 13, Iss. 3. - P. 86-102. DOI: 10.22337/1524-5845-2017-13-3-86-102.

4. Matveev, A.D. Multigrid Finite Element Method in Calculations of Complex-Shaped Composite Beams - 2018. - Part 1. - P. 568-569.

5. Ustimenko E.E., Skachkov S.V. Finite Element Method of a Thin-Walled Profile Model with Shelves of a Volumetric Shaped Element Eng. Vestn. Don. - 2019. - No. 4(55). - P. 54-63.