ABCDEFGHIJKLMNOPQRSTUVWXYZ
1
Windocks subsetting, synthetic data generation, and database movement are CPU and RAM intensive. Configurations vary based on the CPU and RAM available.
2
3
CONFIG SETTINGS
4
Configuration file path
windocks\synthetic\config\config.json
5
Tables parallel
bulkDataMoveAndSubsettingParallelismMaximumTablesToProcessInParallel
6
Batches parallel
bulkDataMoveAndSubsettingParallelismMaximumBatchesForEachTableToProcessInParallel
7
Batch size
bulkDataMoveBatchSizeForReadingRows
8
Transforms in parallel
bulkDataMoveAndSubsettingParallelismMaximumTransformsToProcessInParallel
9
10
Parameters and Model
11
12
# Cores available for Windocks serviceCores
13
Multiplier for cores to get max threads for parallelsimCore Multiplier (2 to 3)
Use a higher setting for slow disk and I/O, allowing more threads to not be blocked or waiting
14
Tables parallel * batches parallelCore multiplier * Cores
15
Tables parallel 1 to 10Lots of tables of similar size suggests more tables in parallel (8 to 10)
16
A database with one table with a disproportionately large row count use a value of 2
17
Batches parallel
(Core multiplier * Cores) / (Tables parallel)
A few tables with massive rows and others much smaller, then use larger batch parallel number
18
19
Transforms in parallel1With multiple transforms in parallel, divide the Tables by this number
20
21
RAM in GB available for Windocks serviceRAM
22
Average row size of your daabase in bytesRowSize
23
24
Memory overhead multiplierMemory multiplier
Larger your RAM, lower the multiplier. Use 2 for 12 GB RAM, use 1.5 for 30 GB or more.
25
26
Batch sizeRAM * 1,000,000,000 / (Memory multiplier * RowSize * Cores * Core multiplier)
27
28
29
EXAMPLE
30
31
# Cores available for Windocks service16
32
Multiplier for cores to get max threads for parallelsim3
33
Tables parallel * batches parallel48
34
Tables parallel 4
35
Batches per table parallel 12
36
37
Transforms in parallel for large databases1
38
39
GB of RAM available for Windocks service12
40
Average row size of your daabase in bytes200
41
42
Memory overhead multiplier2
43
44
Batch size (bulkDataMoveBatchSizeForReadingRows)625,000
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100