No More Data Staging and Power of Batch Apex in salesforce

Data Staging :

A staging area, or landing zone, is an intermediate storage area used for data processing during the extract,
transform and load (ETL) process. The data staging area sits between the data source(s) and the data target(s),
which are often data warehouses, data marts or other data repositories.

In salesforce , we may need data stagging some times before completely process the data from out side and to insert into salesforce objects.

Scenario :

In some situations , we may need to upload large volume of data using file upload process , review the data before process and then upload the data in to salesforce objects.
When processing large volume of the records in apex, some time apex can hit governor limits .

Governors and Limits

Because Apex runs in a multitenant environment, the Apex runtime engine strictly enforces limits to ensure that
runaway Apex doesn’t monopolize shared resources. If some Apex code ever exceeds a limit,
the associated governor issues a run-time exception that cannot be handled.


Power of Batch APEX :

1.Will this works on millions of records ?
Yes batch apex are asynchronous process and can support upto 50 million records and hence for 1 million this code will run

2.Does Force.com run time calls the "execute" method each time with 50k records
You will specify batch size while you invoke batch .Say i specify batch size as 400.Then 50000/400 number of batches will run.In each batch context 400 records will picked .

3.Does the execute method has a time-out after X seconds?
No there are no time outs.There is some time out limit 60000 milliseconds of CPU time for async transactions.

4.Can we control the amount of records being delivered to the "execute" method? (for example if we wish to deliver each time 500 contacts)
Yes we can control this through the scheduler script by specifying the batch size

    String query = 'select Id from Contact'
    ManageRecords batchpr = new ManageRecords (query);
    Id batchprocessId = Database.executeBatch(batchpr,500);
The above script that you have i have split and 500 is my batch size .

Batch Apex Skeleton :

global class SearchAndReplace implements Database.Batchable<sObject>{

   global final String Query;
   global final String Entity;
   global final String Field;
   global final String Value;

   global SearchAndReplace(String q, String e, String f, String v){

      Query=q; Entity=e; Field=f;Value=v;
   }

   global Database.QueryLocator start(Database.BatchableContext BC){
      return Database.getQueryLocator(query);
   }

   global void execute(Database.BatchableContext BC, List<sObject> scope){
     for(sobject s : scope){
     s.put(Field,Value); 
     }
     update scope;
    }

   global void finish(Database.BatchableContext BC){
   }
}


Batch Apex & Data Staging :

There is simple trick to avoid the apex governor limits and process the larger amount of records in to salesforce objects from staging .
Batch apex simplifies this staging mechanism in salesforce and it no more requires staging tables before process the data to salesforce objects.

the only thing you need to do is , prepare data and send it to batch apex to process .

Below is the small example :

global class BatchProcessStaggingsdata implements Database.Batchable<ManageRecords.Contactsstagging>{
   global final list<ManageRecords.Contactsstagging> LstStaggingData;
   global final string strSummarylogId ;
   global integer ibatchCount =1;
   global BatchProcessStaggingsdata(list<ManageRecords.Contactsstagging> LstSelectedStaggingData){
     LstStaggingData=LstSelectedStaggingData;
   }
   global list<ManageRecords.Contactsstagging> start(Database.BatchableContext BC){
      return LstStaggingData;
   }
   global void execute(Database.BatchableContext BC, List<ManageRecords.Contactsstagging> scope){
     system.debug('******LstData'+scope);
     
   }
   global void finish(Database.BatchableContext BC){
     
     
   }
}

In the above process , LstSelectedStaggingData holds the staging data and assigning to a list in the batch apex.

Thank you
Haribabu Amudalapalli