PME I J http://polipapers.upv.es/index.php/IJPME International Journal of Production Management and Engineering https://doi.org/10.4995/ijpme.2021.14483 Received: 2020-10-13 Accepted: 2020-12-30 Validation of production system throughput potential and simulation experiment design Standridge, C.a*, Wynne, M.b a School of Engineering, Grand Valley State University, 301 Fulton Avenue West, Grand Rapids, MI 49504, USA. b Yanfeng Automotive Interiors. 1600 Washington Street, Holland, MI 49423, USA. a standric@gvsu.edu, b mjw.michaelwynne@gmail.com Abstract: The throughput potential of a production system must be designed and validated before implementation. Design includes creating product flow by setting the takt time consistent with meeting customer demand per time period and the average cycle time at each workstation being less than the takt time. Creating product flow implies that the average waiting time preceding each workstation is no greater than the takt time. Kingman’s equation for the average waiting time can be solved for the variation component given the utilization and the cycle time. The variation component consists of the variation in the demand and the variation in the cycle time. Given the variation in demand, the maximum allowable variation in cycle time to create flow can be determined. Throughput potential validation is often performed using discrete event simulation modeling and experimentation. If the variation in cycle time at every workstation is small enough to create flow, then a deterministic simulation experiment can be used. An industrial example concerning a tier-1 automotive supplier with two possible production systems designs and various levels of variation in demand assumed is used to demonstrate the effectiveness of throughput validation using deterministic discrete event simulation modeling and experimentation. Key words: Throughput potential validation, Kingman’s equation, Discrete event simulation. 1. Introduction Validating that a production system can meet customer throughput requirements before implementation has long been an important goal. Ferrin, Muller, and Muthler (2005) discuss how discrete event simulation (DES) is uniquely able to support achieving this goal by finding a very good solution that meets system design and operation requirements before implementation. Marvel and Standridge (2009) discuss and illustrate an enhanced lean improvement process that uses DES to validate a proposed future state of a production system before implementation. One particular class of production systems of interest produces a single part type, or perhaps a small family of part types, for delivery as a subassembly to another business, as opposed to a finished product for delivery to a consumer. This can result from a supplier fulfilling a contract for a specified number of parts per day or week with little or no variation in demand. For example, a tier-1 automotive supplier contracts to build a specific number of door handles per week for an automobile manufacturer. A typical structure for such a system is shown in Figure 1. A main cell completes production of the subassembly to be delivered to the customer. The main cell receives subassemblies from one or more feeder cells. A feeder cell may receive a subassembly from another feeder cell. The cells may not be adjacent within a facility. A worker, called a runner, moves subassemblies among the feeder and To cite this article: Standridge, C., Wynne, M. (2021). Validation of production system throughput potential and simulation experiment design. International Journal of Production Management and Engineering, 9(1), 15-23. https://doi.org/10.4995/ijpme.2021.14483 Int. J. Prod. Manag. Eng. (2021) 9(1), 15-23Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International 15 http://creativecommons.org/licenses/by-nc-nd/4.0/ main cells on a fixed route. The maximum time to complete movement through the route once is a specified constant. Figure 1. Typical structure of a business-to-business production system (source: Authors’ drawing). Each cell consists of one or more workstations. Work is performed either automatically by a machine or according to a standard work specification by a human. Thus, cycle time variance is often small. Since the variation in cycle times is small and the variation in demand is also small, the question arises as to whether these variances can be ignored in formulating the DES model and experiment. If possible to do so, a deterministic model and experiment offers advantages over a stochastic model and experiment. Simulation results are not statistical estimates but constant values. Thus, the analysis of such results is much simplified. Only one replicate is required per combination of input values, greatly reducing computational time. Considering a deterministic model and experiment seems reasonable since Pritsker (1989) points out that many industrial models, around 30%, have no random quantities as they are used to evaluate operating procedures which are complex, multivariate and contain non-trivial algorithms. For example, Kleijnen and Standridge (1988) describe the analysis of a flexible manufacturing system using deterministic DES. For each of three operations, either a machine only capable of performing that operation or a flexible machine capable of performing all three operations was chosen. A regression meta-model related lead time in the system (dependent variable) to the number of each of the four types of machines (independent variables). Lead time depended only on the number of machines performing operation two and the number of flexible machines. Thus, the effectiveness of deterministic simulation was shown. What is missing is a systematic way to evaluate whether deterministic DES can be used. A way to determine the maximum variation is developed which is consistent with the lean manufacturing idea of creating flow. In addition, the DES modeling approach for systems with the structure shown in Figure 1 is presented. Application to an existing production system is shown. The application considers two alternative configurations of workstations as well as no variance, a small variance, and a large variance in demand. 2. Background Reducing model and experiment complexity without compromising the validity required to support decision making is a methodological issue that is still being addressed. This work contributes to this ongoing effort in the context of production systems. Askin and Standridge (1993) provide an overview, including a review of methods. For example, Jayaraman and Gunal (1997) discuss the complexities of automotive powertrain manufacturing resulting from the need to assemble hundreds of components which are produced by separate systems or suppliers and then integrated. How DES models and experiments are used to address this complexity is described and illustrated. More recently, Pinheiro et al. (2019) used DES to compare the performance of push, pull, and pull with a CONWIP control in the operation of a production system. They found that the pull system yielded the best performance with respect to meeting customer demand and minimizing lead time. Zupan and Herakovic (2015) discuss how simulation is used to improve a production line after line balancing using traditional methods has been performed. As seen in these examples, DES models most often contain elements represented by random variables such as cycle times and times between arrivals. Simulation experiments must be designed and multiple replicates, usually between 10 and 30, executed for each of the many combinations of input parameter values as discussed by Law (2014) and Kleijnen (2015). In addition, statistical analysis of the simulation results must be performed, the complexity of which depends on the experiment design. For example, Atalan and Dönmez (2020) report the design of a large-scale, full factorial simulation experiment that was used to reduce patient waiting time in an emergency room by about Int. J. Prod. Manag. Eng. (2021) 9(1), 15-23 Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Standridge and Wynne 16 http://creativecommons.org/licenses/by-nc-nd/4.0/ 75% and increase the number of patients seen by about 10%. Aggregate modeling is one way of reducing model complexity. Khan and Standridge (2019) discuss one possibility for aggregate modeling in the context of a production system. The over 100 products produced by the system are combined into one aggregate product with routing between workstations chosen at random. The model is shown to be valid and effective in estimating the single parameter of a CONWIP flow control system (Spearman, et al. 1990). Computational efficiency related to aggregation methods is discussed by Tribastone and Vandin (2018). Schruben (1983) developed the event graph modeling technique to address the complexity of modeling and simulating extremely long production lines such as those found in the semi-conductor industry. Instead of modeling individual parts moving from workstation to workstation, the sequence of events that changes the state of the system is modeled. Such events include the beginning and end of operations at a station that change station status from idle to busy and back again as well as counting the number of parts at each station. Thus, the number of parts in the system and in any subarea of the system can be directly determined. The average lead time can be computed using Little’s Law (Little, 1961). This approach was shown to greatly reduce simulation execution times. As discussed above, deterministic simulation experiments are less complex than stochastic experiments. For example, Dagkakis et al. (2019) demonstrate the effectiveness of deterministic simulation in analyzing a production system where the experiment uses an optimization algorithm to search the solution space to maximize throughput. This approach optimally assigns cross-trained operators in an assembly line considering the various levels of cross-training of each operator and the demand placed on the line. Improvement in throughput versus assigning workers to minimizes bottlenecks and WIP-levels was shown. Using optimization algorithms to search a simulation experiment solution space typically causes many combinations of model parameter values to be evaluated. Thus, it is important to use deterministic simulation to minimize execution time. What is missing in the published literature is a systematic way to decide when deterministic simulation can be used. While Uriarte et al. (2020) provide a comprehensive review of the joint use of lean and DES, they do not identify reduction in simulation experiment complexity as a research gap. As reduction in variance is a primary goal of lean (Tapping et al. 2002), the opportunity to use deterministic simulation experiments when jointly applying simulation and lean would seem to exist. Simularly, Mourtzis (2019) provides a review of the use of simulation in the design and operation of production systems particularly with regard to Factory 4.0. However, the use of deterministic simulation and complexity reduction in simulation experimentation are not discussed. In the same vein, Sanchez et al. (2020) discuss the design of simulation experiments. While acknowledging the possibility of both deterministic and stochastic experiments, they do not discuss criteria for choosing between them. Finally, Puvanasvaran et al. (2020) discuss a simulation application for determining the throughput of a production systems with significant material movement delays. However, they assume that a stochastic simulation experiment is needed without assessing the possibility of employing a deterministic one. Thus, an approach for determining when deterministic simulation is appropriate for a commonly occurring type of production system as developed here is another, needed step toward reducing DES modeling and experimentation complexity. 3. Methods The simulation modeling approach for a production system with the structure shown in Figure 1 is given in Figure 2. The model consists of one process for the main lines and for each of the feeder cell as well as a process for the runner. The processes do not directly communicate with each other. Instead they share a set of state variables that record the number of the various work-in-process elements, such as subassemblies, in the system. Each feeder cell process produces one or more work-in-process elements and may need one or more work-in-process elements produced by another feeder process to begin working. The main process needs one or more work-in-process elements to begin working and produces one item to finish goods inventory. The runner process moves the work-in-process elements among the feeder lines and the main line. Int. J. Prod. Manag. Eng. (2021) 9(1), 15-23Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Validation of production system throughput potential and simulation experiment design 17 http://creativecommons.org/licenses/by-nc-nd/4.0/ Figure 2. Modeling approach for a constant rate production system (source: Authors’ drawing). An entity arrives to each feeder cell process and the main process at an average time between arrivals equal to the takt time. The runner arrives to the runner process once, at the beginning of the simulation. The takt time is the pace at which work must be done in order to meet the production target for a given time period and is shown mathematically in Equation 1 (Tapping et al., 2002). takt time= work time available (1) number of units required Thus, the average cycle time at each station in each line cannot exceed the takt time. This implies that the average time between initiation of parts on each line is the takt time. This also implies that the average waiting time for processing in the queue at each station cannot exceed the takt time as waiting can be viewed as one more processing step. The average waiting time is given by Kingman’s equation (Kingman, 1961). (2) Each quantity in Equation 2 is described in Table 1. Table 1. Quantities in Kingman’s equation (source: Authors). Quantity Description LTq Average waiting time in the queue preceding a workstation CV Coefficient of variation = standard deviation average CV2 Squared coefficient of variation TBA Time between arrivals CT Cycle time m Workstation utilization = percent busy time = CT / TBA Note that the maximum value for LTq that is consistent with using deterministic simulation is the takt time. In addition, the cycle time can be rewritten as the product of the utilization and the time between arrivals. These lead to Equation 3. (3) Solving for V yields Equation 4. (4) Equation 4 means that in order for deterministic simulation to be used the variation at each workstation can be no greater than a value that is a function of the utilization alone. The difficulty is that the variation, V, consists of two parts: the variation in the time between arrivals and the variation in the cycle time. The strategy will be to assume a value for the variance of the time between arrivals to the first station in the main cell or a feeder cell. As discussed above, this variance should be small when the item produced is to be delivered to another business. The variance could be zero if the demand per week is a constant value specified by a contract. As Standridge and Heltne (2000) discovered, even weekly order sizes placed by another business only have a small random component, no more than 20% of the average order size (CV = 0.2). With the variance of the demand assumed and demand expressed as the time between arrivals, the maximum variance in the cycle time can be computed given the workstation utilization as shown in Equation 5. (5) The maximum allowed value for the variance of the cycle time, expressed as the squared coefficient of variation, is illustrated for the case where the variance of the time between arrivals is low (CVTBA = 0.0, 0.2) as well as for the case where the variance of the time between arrivals is high (CVTBA = 1.0). As previously discussed, the former corresponds to production meeting the demand of another business. The latter corresponds to meeting consumer demand, the practical worse case variance equal to the variance of an exponential distribution as discussed by Hopp and Spearman (2011). The results are shown graphically in Figure 3 and in Int. J. Prod. Manag. Eng. (2021) 9(1), 15-23 Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Standridge and Wynne 18 http://creativecommons.org/licenses/by-nc-nd/4.0/ Table 2. Note that negative values of CV2CT indicate that it is not feasible to use deterministic simulation for a workstation with the corresponding utilization and CVTBA combination. Figure 3. Maximum allowable cycle time variation (source: Author generated). Table 2. Maximum allowable cycle time variation (source: Author generated). utilization CV2CT with CVTBA = 0.0 CV2 CT with CVTBA = 0.2 CV2 CT with CVTBA = 1.0 50% 4.00 3.96 3.00 55% 2.98 2.94 1.98 60% 2.22 2.18 1.22 65% 1.66 1.62 0.66 70% 1.22 1.18 0.22 73% 1.01 0.97 0.01 75% 0.89 0.85 -0.11 80% 0.63 0.59 -0.38 85% 0.42 0.38 -0.58 90% 0.25 0.21 -0.75 95% 0.11 0.07 -0.89 99% 0.02 -0.02 -0.98 While the variance of the time between arrivals to the first workstation in sequence is assumed, the variance of the time between arrivals to the next and following workstations must be determined. This issue is discussed by Hopp and Spearman (2011). The variance in the time between arrivals to the following workstation can be assumed to be equal to the variance in the time between departures from the current workstation, which is a function of the variance in the time between arrivals, the variance in the cycle time, and the utilization as given in Equation 6. (6) Note that even in the case where the variance of the time between arrivals is zero, the variance in the time between departures will be greater than zero if the variance in the cycle time is greater than zero. Based on Equations 5 and 6, the observed variance in the cycle time at each station must be less than the maximum allowed in order to support the use of deterministic simulation. 4. Results and Example Consider an existing parts manufacturing system as shown in Figure 4 with the structure given in Figure 1. Figure 4. Structure of parts manufacturing system (source: Authors’ drawing). The assembly cell produces a final product which consists of the subassemblies produced by PT and HPW, which are carried to the assembly cell by the runner. MAG provides a subassembly to HPW. MAG and HPW are located near enough to each other to allow the direct transfer of the subassembly produced by MAG. Takt time is computed as shown in Table 3. Recall that the takt time is the quotient of the available work time per day, 1305 minutes, and the demand per day, 840 parts. The supplier uses a 10% allowance to control for unforeseen events such as breakdowns, thus lowering the takt time from 93.2 seconds to 83.9 seconds per part. The average cycle time and observed coefficient of variation were estimated from 10 observations of the cycle time for each work element at each workstation. These were collected by manual observation. Int. J. Prod. Manag. Eng. (2021) 9(1), 15-23Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Validation of production system throughput potential and simulation experiment design 19 http://creativecommons.org/licenses/by-nc-nd/4.0/ Table 3. Takt time determination (source: Authors’ data collection). Item Value Units Demand / day 840 parts Work time / shift 435 min Shifts / day 3 Work time / day 1305 min Takt time 1.55 min Takt time 93.2 sec Allowance 10% Takt time with allowance 83.9 sec Table 4 shows the utilization of each workstation in each cell along with the cycle time average and squared coefficient of variation computed from the collected data and the takt time. In addition, the maximum squared coefficient of variation allowed for deterministic simulation computed using Equations 5 and 6 is shown for three values of the coefficient of variation of the time between arrivals to the first station in the cell: CVTBA= 0.0, 0.2, and 1.0. For every station and each CVTBA value, the squared coefficient of variation of the cycle time (CV2) is much less, by orders of magnitude, than the maximum allowed squared coefficient of variation (Max CV2). Thus, deterministic simulation is appropriate regardless of the variation in the time between arrivals. Thus, the simulation model has the structure shown in Figure 5, which follows from Figure 2. Figure 5. Simulation model structure for the parts manufacturing system (source: Authors’ drawing). There are five processes, one for each of the cells and one for the runner. The assembly process waits for a subassembly from the both the PT and HPW processes before proceeding to produce one unit of finished goods inventory. The PT and MAG processes use readily available raw material which is not modeled. The HPW process uses one subassembly from the MAG process. The runner’s route is from Assembly to PT to HPW and back to Assembly. The runner completes one loop in 15 minutes. Five minutes are allocated for each of the three stations: Assembly, PT, and HPW which includes walking between stations, delivering subassemblies, and picking up subassemblies to move. Thus, a station must have a WIP inventory of at least 11 subassemblies (= 15 minutes / 83.9 seconds and rounded up) to avoid starvation between arrivals of Table 4. Variation analysis for deterministic simulation (source: Authors’ analysis). Cell Station Utilization Cycle Time Average CV2 Max CV2 CVTBA 0.0 0.2 1.0 PT PT1 0.48 40.42 0.011 4.47 4.43 3.47 PT2 0.65 54.90 0.0048 1.61 1.59 1.04 HPW HPW 0.71 59.85 0.010 1.13 1.09 0.13 Mag Mag 0.53 44.82 0.011 3.26 3.22 2.26 Assembly ML1A 0.70 58.80 0.0057 1.22 1.18 0.22 ML1B 0.81 67.61 0.0053 0.59 0.58 0.24 ML2 0.65 54.89 0.073 1.59 1.56 1.01 ML3 0.81 68.27 0.0022 0.56 0.55 0.22 ML4 0.91 76.10 0.0094 0.22 0.21 0.04 EOL 0.69 57.65 0.0070 1.32 1.30 0.79 Audit 0.52 43.72 0.017 3.52 3.49 2.80 Int. J. Prod. Manag. Eng. (2021) 9(1), 15-23 Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Standridge and Wynne 20 http://creativecommons.org/licenses/by-nc-nd/4.0/ the runner. However, the subassemblies are organized into totes with capacity 8. At least two totes, 16 subassemblies, are needed to avoid starvation. This is the initial value of the WIP_PT and WIP_HPW at the assembly station as well as WIP_MAG which is shared between the MAG and HPW stations. Furthermore, at the beginning of the simulation, there is one tote in WIP_PT at the PT station and one tote in WIP_HPW at the HPW station. Thus, the simulation begins with a total WIP inventory of 64 subassemblies. The key results from the deterministic simulation are as follows: 1. The utilization reported by the simulation for each workstation is exactly the same as shown in Table 4. 2. The number of finished units produced in one week (5 days) is 4662, which exceeds the demand of 4200 parts. 3. The average WIP inventory in the entire production system is 75 subassemblies and the maximum 76 subassemblies. An alternative configuration of the production system has been proposed under which all cells are co-located so that the runner is not needed. In addition, the work elements for the stations comprising the assembly cell are consolidated such that the Audit station is no longer needed. Further, the work elements for the PT cell are improved such that only one workstation is needed. The utilization and variation analysis for this configuration are shown in Table 5. Note that while deterministic simulation is feasible for CVTBA = 0.0 and 0.2, it is not feasible for CVTBA= 1.0. The maximum allowed value of CV2CT for PT is less than zero and CV2CT for ML4 is greater than the maximum allowed value. The assembly cell must have enough WIP to avoid starvation. In the proposed configuration, one tote of each type of subassembly, those from PT and HPW should be sufficient. In the same way, the HPW cell needs one tote of subassemblies from the MAG cell. Thus, the simulation begins with a total WIP inventory of 24 subassemblies. The key results from the deterministic simulation are as follows: 1. The utilization reported by the simulation for each workstation is exactly the same as shown in Table 5. 2. The number of finished units produced in one week (5 days) is 4666, which exceeds the demand of 4200 parts. 3. The average WIP inventory in the entire production system is 33 subassemblies and the maximum 34 subassemblies. 5. Discussion Kingman’s equation has been widely used to analyze production systems as discussed in Hopp and Spearman (2011) and Standridge (2019). The variation term in Kingman’s equation has two components: the variation in cycle time and in the time between arrivals (demand). Previous work is extended by determining the level of variation in Table 5. Variation analysis for deterministic simulation – Consolidated workstations (source: Authors’ analysis). Cell Station Utilization Cycle Time Average CV2 Max CV2 CVTBA 0.0 0.2 1.0 PT PT 0.78 65.60 0.17 0.71 0.67 -0.29 HPW HPW 0.71 59.85 0.010 1.13 1.09 0.13 Mag Mag 0.53 44.82 0.011 3.26 3.22 2.26 Assembly ML1A 0.72 60.18 0.09 1.10 1.06 0.10 ML1B 0.82 69.12 0.07 0.48 0.46 0.15 ML2 0.84 70.22 0.05 0.43 0.42 0.13 ML3 0.85 70.96 0.05 0.39 0.38 0.11 ML4 0.86 72.14 0.11 0.30 0.29 0.04 EOL 0.84 70.63 0.08 0.39 0.38 0.10 Int. J. Prod. Manag. Eng. (2021) 9(1), 15-23Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Validation of production system throughput potential and simulation experiment design 21 http://creativecommons.org/licenses/by-nc-nd/4.0/ cycle time that is consistent with average waiting time preceding a workstation being less than the takt time given the variation in the time between arrivals. This condition indicates that flow in the production system has been achieved, one of the chief goals of lean. Furthermore, this is the condition under which deterministic simulation can be used to validate production throughput potential. The practical worst case, CVTBA = 1, models demand coming from consumers as opposed to another business. As seen in Figure 3 and Table 2 for this case, the maximum allowed squared coefficient of variation is positive for workstations with utilizations of 73% and less. Thus, low utilization workstations can be modeled as deterministic if the variation in the cycle time is low enough which is likely the case for a utilization of 65% or less. To model demand coming from another business, CVTBA = 0.0 or 0.2 was used. In this case, workstations with utilizations up to 90% and reasonable small variation can be modeled using deterministic simulation as can workstations with utilizations between 90% and 99% having very small cycle time variation. Also note that the maximum allowed variation declines in a non-linear fashion as the utilization increases. The use of deterministic simulation versus stochastic simulation greatly simplifies experimentation and results analysis as well as reducing computer execution time. The example demonstrates the effectiveness of this approach. The model is validated by comparing the utilization of each workstation computed from gather data with that computed by the simulation. These quantities were found to be the same for each workstation for each configuration. In addition, each configuration was shown to be capable of meeting the throughput target set by the customer with near constant work-in-process inventory as the average and maximum values differ by only one. The proposed reconfiguration results in reducing the WIP by over 50% as well as eliminating the runner and two workstations. Thus, it is preferred. 6. Conclusions Criteria for using deterministic simulation of production systems versus stochastic simulation are established. This adds to the existing literature both of the use of deterministic simulation and to the application of Kingman’s equation to production systems. The criteria are straightforward to apply requiring only the comparison of a quantity that is a function of workstation utilization to the known or estimated variation in the cycle time at each workstation. Both are expressed as the squared coefficient of variation. References Atalan, A., Dönmez, C.C. (2020). Optimizing experimental simulation design for the emergency departments. Brazilian Journal of Operations & Production Management, 17(4), e2020854. https://doi.org/10.14488/BJOPM.2020.026 Askin, R.G., Standridge, C.R. (1993). Modeling and analysis of manufacturing systems. New York: John Wiley and Sons. Dagkakis, G., Rotondo, A., Heavey, C. (2019). Embedding optimization with deterministic discrete event simulation for assignment of cross-trained operators: an assembly line case study. Computers and Operations Research, 111, 99-115. https://doi.org/10.1016/j. cor.2019.06.008 Ferrin, D.M., Miller M.J., Muthler D. (2005). Lean sigma and simulation, so what’s the correlation?, in Proceedings of the 2005 Winter Simulation Conference, IEEE, USA. Retrieved July 22, 2020 from: https://informs-sim.org/wsc05papers/249.pdf Hopp, W.J., Spearman, M.L. (2011). Factory Physics: Foundations of manufacturing management, 3rd ed. Long Grove, IL: Waveland Press. Jayaraman, A., Gunal, A.K. (1997). Applications of discrete event simulation in the design of automotive powertrain manufacturing systems”. In Proceedings of the 1997 Winter Simulation Conference, IEEE, USA. https://doi.org/10.1145/268437.268620 Khan, S., Standridge, C.R. (2019). Aggregate simulation modeling with application to setting the CONWIP limit in an HMLV cell. International Journal of Industrial Engineering Computation, 10(2), 149-160. https://doi.org/10.5267/j.ijiec.2018.10.002 Kingman, J.F.C. (1961). The single server queue in heavy traffic. Mathematical Proceedings of the Cambridge Philosophical Society, 57(4), 902. https://doi.org/10.1017/S0305004100036094 Kleijnen, J.P.C. (2015). Design and analysis of simulation experiments. New York: Springer. https://doi.org/10.1007/978-3-319-18087-8 Kleijnen, J.P.C., Standridge, C.R. (1988). Experimental design and regression analysis in simulation: an FMS case study. European Journal of Operations Research, 33, 257-261. https://doi.org/10.1016/0377-2217(88)90168-3 Law, A.M. (2014). Simulation modeling and analysis, 5th ed. New York: McGraw-Hill. Int. J. Prod. Manag. Eng. (2021) 9(1), 15-23 Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Standridge and Wynne 22 https://doi.org/10.14488/BJOPM.2020.026 https://doi.org/10.1016/j.cor.2019.06.008 https://doi.org/10.1016/j.cor.2019.06.008 https://informs-sim.org/wsc05papers/249.pdf https://doi.org/10.1145/268437.268620 https://doi.org/10.5267/j.ijiec.2018.10.002 https://doi.org/10.1017/S0305004100036094 https://doi.org/10.1007/978-3-319-18087-8 https://doi.org/10.1016/0377-2217(88)90168-3 http://creativecommons.org/licenses/by-nc-nd/4.0/ Little, J.D.C. (1961). A proof for the queuing formula: L = λW. Operations Research, 9(3), 383-387. https://doi.org/10.1287/opre.9.3.383 Marvel, J.H., Standridge, C.R. (2009). A simulation enhanced lean design process. Journal of Industrial Engineering and Management, 2(1), 90-113. https://doi.org/10.3926/jiem.2009.v2n1.p90-113 Mourtzis, D. (2019) Simulation in the design and operation of manufacturing systems: state of the art and new trends. International Journal of Production Research, 58(7), 1927-1949. https://doi.org/10.1080/00207543.2019.1636321 Pinheiro, N.M.G, Cleto, M.G., Zattar, I.C., Muller, S.I.M.G. (2019). Performance evaluation of pulled, pushed and hybrid production through simulation: a case study. Brazilian Journal of Operations & Production Management, 16, 685-697. https://doi.org/10.14488/BJOPM.2019.v16.n4.a Pritsker, A.A.B. (1989). Why simulation works. In Proceedings of the 1989 Winter Simulation Conference, IEEE, USA. https://doi.org/10.1145/76738.76739 Puvanasvaran, P., Teoh, Y.S., Ito, K. (2020). Novel availability and performance ratio for internal transportation and manufacturing processes in job shop company. Journal of Industrial Engineering and Management, 13(1), 1-17. https://doi.org/10.3926/jiem.2755 Sanchez, S.M., Sanchez, P.J., Wan, H. (2020). Work smarter, not harder: a tutorial on designing and conducting simulation experiments. In Proceedings of the 2020 Winter Simulation Conference, IEEE, USA. Retrieved December 23, 2020 from https://informs-sim.org/ wsc20papers/135.pdf Schruben, L. (1983). Simulation modeling with event graphs. Communications of the A.C.M., 26(11). https://doi.org/10.1145/182.358460 Spearman, M.L., Woodruff, D.L., Hopp, W.J. (1990). CONWIP: A pull alternative to Kanban, International Journal of Production Research, 28(5), 879-894. https://doi.org/10.1080/00207549008942761 Standridge, C.R. (2019). Introduction to production: philosophies, flow, and analysis. Allendale Michigan: Grand Valley State University Libraries. Retrieved July 22, 2020 from: https://scholarworks.gvsu.edu/books/22/ Tapping, D., Luyster, T., Shuker, T. (2002). Value stream management. Boca Raton: CRC Press. https://doi.org/10.4324/9781482278163 Tribastone, M., Vandin, A. (2018). Speeding up stochastic and deterministic simulation by aggregation: an advanced tutorial. . In Proceedings of the 2018 Winter Simulation Conference, IEEE, USA. https://doi.org/10.1109/WSC.2018.8632364 Uriarte, A.G., Ng, A.H.C., Moris, M.U. (2020). Bringing together Lean and simulation: a comprehensive review, International Journal of Production Research, 58(1), 87-117. https://doi.org/10.1080/00207543.2019.1643512 Zupan, H., Herakovic. N. (2015). Production line balancing with discrete event simulation: a case study”, IFAC-PapersOnLine, 48(3), 2305- 2311. https://doi.org/10.1016/j.ifacol.2015.06.431 Int. J. Prod. Manag. Eng. (2021) 9(1), 15-23Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Validation of production system throughput potential and simulation experiment design 23 https://doi.org/10.1287/opre.9.3.383 https://doi.org/10.3926/jiem.2009.v2n1.p90-113 https://doi.org/10.1080/00207543.2019.1636321 https://doi.org/10.14488/BJOPM.2019.v16.n4.a https://doi.org/10.1145/76738.76739 https://doi.org/10.3926/jiem.2755 https://informs-sim.org/wsc20papers/135.pdf https://informs-sim.org/wsc20papers/135.pdf https://doi.org/10.1145/182.358460 https://doi.org/10.1080/00207549008942761 https://scholarworks.gvsu.edu/books/22 https://doi.org/10.4324/9781482278163 https://doi.org/10.1109/WSC.2018.8632364 https://doi.org/10.1080/00207543.2019.1643512 https://doi.org/10.1016/j.ifacol.2015.06.431 http://creativecommons.org/licenses/by-nc-nd/4.0/