Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
lesson_22 [2019/08/23 13:39]
argemiro
lesson_22 [2020/02/19 14:16] (current)
argemiro
Line 3: Line 3:
 \\ \\
 \\ \\
-====LESSON 19:  ​Heuristic calibration of models by using Genetic Algorithm ​ ====+=====Heuristic calibration of models by using Genetic Algorithm  ​=====
    
  
Line 22: Line 22:
 In the example provided, we make available an application of GA tool to calibrating a model of deforestation. The aim of this exercise is to demonstrate the potential of GA tool for calibrating any type of model-as long as there is sufficient computer power to run a model together with its evaluation function numerous times-as well as the overspecialization problem that may arise when using such hard predictors, as artificial neural network and GA itself.  ​ In the example provided, we make available an application of GA tool to calibrating a model of deforestation. The aim of this exercise is to demonstrate the potential of GA tool for calibrating any type of model-as long as there is sufficient computer power to run a model together with its evaluation function numerous times-as well as the overspecialization problem that may arise when using such hard predictors, as artificial neural network and GA itself.  ​
  
-First, load the model ''​Cal_Reciprocal_fitness1x1.ego''​ from ''​\ ​Examples\ Genetic_Algorithm\WEofE''​. This model calculates the reciprocal fitness of a deforestation model that was calibrated using the Weights of Evidence method – a soft predictor. Run the model and access its output ''​Reciprocal_fitness1x1.csv''​ using either a spreadsheet or enabling table viewer on the output port of Set Key 1. The concept of this validation measure is provided in sixth  and seventh chapters of lesson [[tutorial:​building_a_land-use_and_land-cover_change_simulation_model|Building a Land use and Land-cover Change Simulation Model]]. The fitness obtained for this model is 0.2060. Open the Weights of Evidence tables in ''​originals\tables''​ and their corresponding maps in ''​originals\maps''​ under the folder ''​Genetic_Algorithm''​. In this model, //​[[:​Calculate Map]]// replaces //​[[:​calc_w._of_e._probability_map|Calc W. E. Probability Map]]//. Open it to see the equation that integrates the weights of evidence to produce the transition probability map. The weights of evidence coefficients are input as separate tables, so they can form a group of table and thereby the gene.+First, load the model ''​Cal_Reciprocal_fitness1x1.ego''​ from ''​\Guidebook_Dinamica_5\Models\Genetic_Algorithm\WEofE''​. This model calculates the reciprocal fitness of a deforestation model that was calibrated using the Weights of Evidence method – a soft predictor. Run the model and access its output ''​Reciprocal_fitness1x1.csv''​ using either a spreadsheet or enabling table viewer on the output port of Set Key 1. The concept of this validation measure is provided in sixth  and seventh chapters of lesson [[tutorial:​building_a_land-use_and_land-cover_change_simulation_model|Building a Land use and Land-cover Change Simulation Model]]. The fitness obtained for this model is 0.2060. Open the Weights of Evidence tables in ''​\Guidebook_Dinamica_5\Database\Genetic_algorithm\tables''​ and their corresponding maps in ''​\Guidebook_Dinamica_5\Database\Genetic_algorithm\maps''​. In this model, //​[[:​Calculate Map]]// replaces //​[[:​calc_w._of_e._probability_map|Calc W. E. Probability Map]]//. Open it to see the equation that integrates the weights of evidence to produce the transition probability map. The weights of evidence coefficients are input as separate tables, so they can form a group of table and thereby the gene.
  
-Now, open ''​GAReciprocal_fitness1x1.ego''​ from ''​Genetic_Algorithm\GAknn\Reciprocal_fitness1x1''​. Compare the structure of this model with the diagram from Fig.1. Open GA tool. This container envelops three //​[[:​Group]]//​s and two functors. //[[:Get Current Individual]]//​ obtains the gene of an individual pertaining to a generation and passes it to a sequence of functors that extract the lookup tables that compose the gene. A land change simulation model receives those tables as input and its execution results are passed to a fitness function that assesses its spatial performance. In turn, this function returns the fitness measure that is caught and passed to GA tool by //[[:Set Fitness]]//​. ​+Now, open ''​GAReciprocal_fitness1x1.ego''​ from ''​\Guidebook_Dinamica_5\Models\Genetic_Algorithm\GAknn\Reciprocal_fitness1x1''​. Compare the structure of this model with the diagram from Fig.1. Open GA tool. This container envelops three //​[[:​Group]]//​s and two functors. //[[:Get Current Individual]]//​ obtains the gene of an individual pertaining to a generation and passes it to a sequence of functors that extract the lookup tables that compose the gene. A land change simulation model receives those tables as input and its execution results are passed to a fitness function that assesses its spatial performance. In turn, this function returns the fitness measure that is caught and passed to GA tool by //[[:Set Fitness]]//​. ​
  
 {{ :​tutorial:​ga_2.jpg |}} {{ :​tutorial:​ga_2.jpg |}}
Line 36: Line 36:
 <​note>​**Note**:​ Model fitness can be increased by lowering **Prune Factor** in [[:​Patcher]]. This diminishes model stochasticity,​ but often results in less realism.</​note> ​   <​note>​**Note**:​ Model fitness can be increased by lowering **Prune Factor** in [[:​Patcher]]. This diminishes model stochasticity,​ but often results in less realism.</​note> ​  
  
-Open ''​validation_fitness1x1.ego''​ from ''​Genetic_Algorithm\GAknn\Reciprocal_fitness1x1''​ and run it. What is the fitness now? Surprised? What happened? ​+Open ''​validation_fitness1x1.ego''​ from ''​\Guidebook_Dinamica_5\Models\Genetic_Algorithm\GAknn\Reciprocal_fitness1x1''​ and run it. What is the fitness now? Surprised? What happened? ​
  
 The answer is simple. GA tool attained a very high fitness in the calibration process through overspecialization. That is, it adapted so well to this specific situation given by the map of changes from 1997 to 2000, that the probability map obtained with GA tool results became useless when applied to model validation, which is performed by using deforestation from 2000 to 2003. Thus, although GA tool can achieve high scores in the calibration process, it will become overspecialized if the formation of genes in the reproduction process (mostly through mutation and crossover) is not constrained within a certain range of variation from the characteristics of the primeval individual. The answer is simple. GA tool attained a very high fitness in the calibration process through overspecialization. That is, it adapted so well to this specific situation given by the map of changes from 1997 to 2000, that the probability map obtained with GA tool results became useless when applied to model validation, which is performed by using deforestation from 2000 to 2003. Thus, although GA tool can achieve high scores in the calibration process, it will become overspecialized if the formation of genes in the reproduction process (mostly through mutation and crossover) is not constrained within a certain range of variation from the characteristics of the primeval individual.
Line 42: Line 42:
 Dinamica EGO´s GA tool provides a means to overcome overspecialization by enabling the user to specify an envelope of maximum variation to the new genes that will be formed during the reproduction process. So, instead of default values, this envelop is delimited by two group tables, consisting, respectively,​ of the upper and lower gene bounds. Dinamica EGO´s GA tool provides a means to overcome overspecialization by enabling the user to specify an envelope of maximum variation to the new genes that will be formed during the reproduction process. So, instead of default values, this envelop is delimited by two group tables, consisting, respectively,​ of the upper and lower gene bounds.
  
-Open model ''​GAReciprocal_fitness1x1.ego''​ from ''​Genetic_Algorithm\GALimitedRanges120\Reciprocal_fitness1x1''​ and examine //​[[:​Group]]//​ "Lower Bound Envelope"​. Open it. Each //​[[:​Calculate Lookup Table]]// establishes a lower bound for a weights of evidence table by using the following equation:+Open model ''​GAReciprocal_fitness1x1.ego''​ from ''​\Guidebook_Dinamica_5\Models\Genetic_Algorithm\GALimitedRanges120\Reciprocal_fitness1x1''​ and examine //​[[:​Group]]//​ "Lower Bound Envelope"​. Open it. Each //​[[:​Calculate Lookup Table]]// establishes a lower bound for a weights of evidence table by using the following equation:
  
 **t1[line] - t1[line] * 1.2** **t1[line] - t1[line] * 1.2**
Line 50: Line 50:
 **t1[line] + t1[line] * 1.2** **t1[line] + t1[line] * 1.2**
  
-The use of these tables as gene bounds will constrain the new genes to an envelope of ±1.2 times the values of the original weights of evidence coefficients,​ thus providing a trend around which the global optimum solution must be found. Although this constraint will result in lower calibration scores (Run ''​validation_fitness1x1.ego''​ located in ''​GALimitedRanges120%\Reciprocal_fitness1x1''​),​ it will tame the //GA tool// engine, allowing it to improve the Weights of Evidence result for application to a general prediction process (Figs. 2 and 3).    ​+The use of these tables as gene bounds will constrain the new genes to an envelope of ±1.2 times the values of the original weights of evidence coefficients,​ thus providing a trend around which the global optimum solution must be found. Although this constraint will result in lower calibration scores (Run ''​validation_fitness1x1.ego''​ located in ''​\Guidebook_Dinamica_5\Models\Genetic_Algorithm\GALimitedRanges120\Reciprocal_fitness1x1''​),​ it will tame the //GA tool// engine, allowing it to improve the Weights of Evidence result for application to a general prediction process (Figs. 2 and 3).    ​
  
 In conclusion, hard predictors like GA tool must incorporate prior knowledge in order to overcome overspecialization. When this is taken into consideration,​ GA tool can really push the envelope of model optimization. ​ Use your expertise to develop other GA tool calibration processes following the scheme of Fig. 1 and examples provided. In conclusion, hard predictors like GA tool must incorporate prior knowledge in order to overcome overspecialization. When this is taken into consideration,​ GA tool can really push the envelope of model optimization. ​ Use your expertise to develop other GA tool calibration processes following the scheme of Fig. 1 and examples provided.
Line 60: Line 60:
 [{{ :​tutorial:​ga_ 5.1.jpg |Fig.3- ​ a) Transition probability map from Weights of Evidence (blue: low probability,​ red: high probability),​ b) Simulated changes (red) by using map from a) over historical changes from 1997 to 2000 (black). c) Transition probability map using weights output from //​[[:​Genetic Algorithm Tool]]// limited within a 120% envelope. d) Simulated changes (red) using map from b) over historical changes from 1997 to 2000 (black). e) Weights of evidence of deforestation as a function of distance to previously deforested areas (WofE- Weights of Evidence method, GAknn - GA sole method, others: GA with values limited, respectively,​ within 80% and 120% envelope of original weights of evidence values, ​ f ) Simulated changes (red) using map from c) over historical changes from 2000 to 2003 (black).}}] [{{ :​tutorial:​ga_ 5.1.jpg |Fig.3- ​ a) Transition probability map from Weights of Evidence (blue: low probability,​ red: high probability),​ b) Simulated changes (red) by using map from a) over historical changes from 1997 to 2000 (black). c) Transition probability map using weights output from //​[[:​Genetic Algorithm Tool]]// limited within a 120% envelope. d) Simulated changes (red) using map from b) over historical changes from 1997 to 2000 (black). e) Weights of evidence of deforestation as a function of distance to previously deforested areas (WofE- Weights of Evidence method, GAknn - GA sole method, others: GA with values limited, respectively,​ within 80% and 120% envelope of original weights of evidence values, ​ f ) Simulated changes (red) using map from c) over historical changes from 2000 to 2003 (black).}}]
  
-[[tutorial:​dinamica_ego_script_language_and_console_launcher|Next Session]]+\\ 
 +\\ 
 +===Congratulations,​ you have successfully completed this lesson!=== 
 +\\ 
 +[[lesson_23|Next: Agent Based Model]] 
 +\\ 
 +☞[[:​guidebook_start| Back to Guidebook Start]]