Technical Paper SAS Intelligent Advertising for Publishers ® Simulation Process Overview Contents Simulation Process Overview .................................................................2 Intra-Day Inventory or Real Time Inventory ......................................4 Forecasting Tools ...............................................................................7 Frequently Asked Questions..............................................................9 Use Case ............................................................................................10 Terminology.......................................................................................11 i Simulation Process Overview SAS® Intelligent Advertising for Publishers (SASIA) can simulate future ad traffic based on your actual historical request data. SASIA creates an output table that includes all of the ad serving data for a defined set of time. The raw simulation output is available via a BI report using the “inventory_empty” data cube. After the process is complete, the table is used as the source for all the forecasting tools. The output data purposely looks like your actual historical ad serving data to facilitate accurate comparisons. This process can and should be refined to make the data it creates as relevant to your business as possible. See the included Use Case section for examples of what can be configured in the process. Contact SASIA Technical Support for more information and assistance with the refining process. Target: A combination of key value pairs and Boolean logic that define a reusable rule that can be applied to SASIA system objects (e.g., Products, Flights and Creatives). This rule limits the places (combinations of keys and values) the object can serve and count as ad impressions. Product: A SASIA system object that is a combination of a target and information about how much those impressions should cost an advertiser. Simulation: The process SASIA uses to generate data for calculating Capacity, Availability and Overlap for a specified set of Keys and Values (also known as Targeting or Products). Frequently used interchangeably with Inventory and Forecasting. Inventory: Blanket term for the collection of tools and data used to forecast impression volume for specified sets of Keys and Values Frequently used interchangeably with Forecasting and Simulation. 2 Forecasting: Blanket term for the act of using the SASIA inventory forecasting tools. Frequently used interchangeably with Inventory and Simulation. Capacity: The sum of all the impression served during simulation for a specified set of Keys and Values from all Tiers. Availability: The sum of all the impressions served during simulation for a specified set of Keys and Values from Tiers marked Preemptible. Booked: The sum of impressions served during the simulation from all Tiers from currently Live flights whose targeting matches a specified set of Keys and Values. For more information, refer to the following SAS Notes: • Inventory Projections • Simulation Forecast Methods • Contending Flights 3 Intra-Day Inventory or Real Time Inventory SASIA also provides another backend process that runs in real time (during the day) to keep the available numbers up to date until the simulation can be run again the next night. For example: A simulation runs over night and creates data that states 1,000,000 of 2,000,000 impressions for product X are available. That data is used the next morning to create a proposal and check the inventory. If the proposal is created for 250,000 impressions and the Probability set to 90%, the system will decrement the Avails for Product X appropriately. Then, when another person creates a proposal for 800,000 impressions 45 minutes later, SASIA will report that there are only 750,000 impressions available for that product. Note: The capacity will fluctuate slightly each time the simulation is run since a slightly different set of historical dates is used each time. Specifically, capacity for product X might be reported as 2,000,345 today and may result in 1,999,896 the next day. In most cases differences should be small enough not to affect the general sense of what is available. Historical Basis (e.g., 28 days) This is the amount of time in the past selected from historical request logs that is used as future request data for the simulation. This data contain dates, cookie Ids, and TAG and BT_TRAIT information. SASIA generally selects a number between 30 and 60 for this setting since the tags used in these requests are likely to be similar to the real requests that will come in tomorrow, next week and next month. If the number goes back too far, the risk of the tag data being dissimilar, forcing the system to use estimations on some tags instead of actual data, increases. 4 Projection Period/Forecast Length (e.g., 365 days) This is the amount of time in the future SASIA attempts to predict using the request data from the sampling files (historical request data). For example, if you open the Inventory_Empty analyzer view in BI you will be able to see data that looks like detailed delivery data but for 30, 60, 180, or more days in the future. Keep in mind that the longer the simulation period, the longer it takes the process to run over night, although one year is usually more than enough. Sampling Rate (default is 53) This is the number of requests from the sampling logs SASIA skips for every request used. Using every request in the sampling file makes the simulation process take much longer, and is not that much more accurate. The default setting of 53 produces accurate forecasts. In testing a sampling rate of 53 vs 1 showed no huge difference in calculated availability numbers. Lowering the sampling rate is only recommended when you have many high value, small impression goal flights. System Default Ads The 1x1 pixel ad that gets served by the system when no other flights can serve to the request from any tier. Preemptible Flag A setting on the Tier Edit screen that alerts the simulation process to treat ads served from this tier as available. If you have no tiers set as Preemptible, only the probably small pool system default ads are considered available impressions. For example, you may have several tiers of ads that serve unpaid house banners or redirects to third party monetization services. You generally DO NOT want the system to treat these as booked inventory, which deprives your direct sales team access to these impressions when building proposals. Ignore Preemptible Tells the simulator to treat tiers set with the preemptible flag as normal and to only use system default ads served during simulation in calculating available impressions. 5 For more information, refer to the following SAS Notes: • Preemptible Feature • Configuring Tiers • Flight Statuses • Targeting Criteria that cannot be included in Simulation • Multivalue Targeting • Integrated Passback Handling • Multiple sizes in Ad Calls 6 Forecasting Tools Inventory_Empty A data cube option in the BI reports that gives you direct access to the simulation output table. Should you want to explore the data in order to formulate better questions about your product catalog or delivery patterns, you might want to start here. While you can find detailed capacity and availability numbers here, it is more of a process than our other forecasting tools. Check Avails Screen This forecasting tool is useful in situations where you have a specific question, "What are the Capacity and Availability numbers for Product X"? This information was devoted to its own screen since we discovered customers were creating dummy line items to use the simple check avails button that gave a simple number that was the answer to their question. Check Avails Button This is available on the Flight Edit and Line Item edit screens and allows you to ask if there will be sufficient impressions available for the line item or flight you are currently creating. It makes use of the Start and End dates along with any targeting you have already specified. Product Report The Product Report uses historical data analysis to calculate Projected and Available impression counts for a specified date range for products created in SASIA. The Projected value in the report is a forecast of how much a product could deliver in the forecast date range while the Available value is a forecast of what will actually be available during the forecast date range. Tag Assessment Report The Tag Assessment Report allows you to see what percent of total impressions a particular tag is involved in. This is useful in helping refine what data you actually want in the simulation process. Focusing keys used helps improve accuracy while maintaining performance. This should be used in conjunction with a sampling of flights based on revenue as describe in the use case below: Tag Value Percentage cat 25,156,825.57 100 SITE 22,261,854.18 88 AREA 22,115,339.53 87 state 19,099,631.53 75 7 sub 14,377,359.20 57 pos 13,522,707.91 53 platform 4,562,389.88 18 TARGET 3,005,264.93 11 agent 2,593,200.00 10 USER_AGENT 2,593,200.00 10 price 726,612.88 2 region 428,207.78 1 reg 428,207.78 1 proptype 407,250.74 1 DAYOFWEEK 361,064.93 1 segment 322,459.23 1 geo_region 203,621.94 0 pname 202,300.01 0 space 59,983.32 0 geo_country 55,469.44 0 TIME 51,000.00 0 brand 23,231.00 0 language 6,401.85 0 For more information, refer to the following SAS Notes: • Product Report • Proposal and Line Item Compliance • Available Button and Check Avails Tab 8 Frequently Asked Questions What flights are conflicting for impressions in product X? Can I re-arrange it? This information is available on the contending flights screen: http://support.sas.com/kb/53/570.html Why am I getting different numbers from different ways of accessing inventory? It is possible that the targeting is slightly different in each case. Another possibility is the requirement to estimate data for keys that are not included in the simulation data set. The use case and other sections of this document should help minimize the use of tag estimation. Why are campaigns under delivering despite there being available inventory? The engine wants to deliver a flight at a specific impressions per second rate. If that rate is met, some impressions will go to other flights (maybe even preemptible or defaults, creating available inventory). The engine doesn’t know there is decreasing availability in the future, it simply serves it at the rate it needs to in order to meet the calculated curve. For example, if you have a goal of 1000 and the flight runs for 10 days, ideally you should see about 100 impressions a day. If the availability of those 1000 impressions occurs mainly on one end of the flight’s run time then you will see underdelivery. If all 1000 impressions were available on the first day but NO impressions were available on the subsequent days, you would see about 900 impressions available for that period even though the flight will under deliver. If the availability starts to decrease while the engine wants to increase the serve rate, you will see situations like this. Why is a product that is oversold still giving me avails? Flights not set for Top Priority will allow some inventory to slip through when the Engine (or Simulator) perceives that the flight is meeting whatever rate has been calculated for it to get where it needs to be. For this flight to get as much inventory as it can, it needs to be Top Priority. Short of that, a Delivery Patience of 0.0 will help, but it won’t eliminate leaking to lower tiers, because the available inventory doesn’t present itself evenly. Traffic availability over time may affect the available inventory. For example, the forecast could show 1000 impressions available on a Monday but not on subsequent days; if the flight has a goal of 1000 but is scheduled to be delivered from Monday through Sunday, the 1000 impressions are not going to be available. Note that while there may have been sufficient inventory when the flight was created, the available inventory currently available is affected by subsequent changes (contending flights added, goals increased, end dates shortened, etc.). Why are availability and capacity numbers different from one day to the next? The numbers fluctuate slightly each time the simulation is run since a slightly different set of historical dates is used. Specifically, capacity for product X might be reported as 2,000,345 today and running the same report tomorrow might result in 1,999,896. In most cases differences should be small enough not to affect the general sense of what is available. 9 Use Case In this use case, a thought process is outlined that might improve your SASIA forecasting results. The specific goal of this exercise is to tune and optimize your simulation process towards the Flights, Tags and Products that make the most money and not go over your current capacity. First we need some source data: Using the BI cube BIGSUM_EMPTY find a list of the top 20 revenue generating flights from the last 30 days. Note: You can export to Excel if that is easier to work with. It should look something like this: Product Name Flight Floor Rate Sold Rate Goal Rate List Rate Impress Total Rev Served Main Homepage CPM - 300x250 Honda - June - Home Page - 300x250 - 2015 $9.00 $9.15 $12.00 $15.00 1576414 $14,424.19 Main Homepage CPM - 300x600 Homepage - CPM 300x600 $15.00 $12.50 $20.00 $25.00 1144301 $14,303.76 Main Homepage CPM - 300x600 Birthday Pop for Prizes Homepage - 300x600 30 - 05 - 2015 $15.00 $17.50 $20.00 $25.00 363597 $6,362.95 Motors - CPM 300x250 Honda - June - Motors Home Page - 300x250 2015 $12.00 $9.15 $16.00 $20.00 687681 $6,292.28 Next find all the Tags (also known as Keys) used in the targeting from those flights. We found that 20 standard tags and BT traits showed up in our 20 flight sample. However, before we change the configuration we should consider if the previous 30 days constitute a good sample of flights? For example are there any BT traits that are frequently used that may not have been used during these 30 days? Are there any other special circumstances that might argue for evaluating flights from 60, 90 or 120 days ago? The current simulator configuration has the following Tag details: site, area, geo_country, sc, a24, a8l, a18h, a9l, a49h, c, e, i, topic, ms, retargeting, pos, dev_type, dev_os, msg, body, a34, a35 Standard & BT tags from the top 20 flights: Site, Area, E, Msg, ptype, Dev_type, MS, P, C, Topic, Body, Model, abc, Vehicle, BT_Autos, BT_autos_newcars, BT_autos_over40k, BT_GI _industry, BT_make, BT_phones Recommendation: There seems to be a discrepancy between the tags that the simulation is running with and the tags we seem to need based on the most profitable flights. Ideally we want to have the tags that are commonly used in the flights that provide the most value in the simulation directly. Other tags will use estimates and are consequently not as accurate. Therefore, having an inappropriate set of tags in the Simulation can reduce the accuracy of your most important forecasts. 10 Terminology Key Value Pairs: An item of data that is identified by an arbitrary name. The key is the name, and the value is the content; for example, the key might be AGE, and the value might be 25. Target: A combination of Key value pairs and Boolean logic that define a reusable rule that can be applied to SASIA system objects; for example, Products, Flights and Creatives. This “rule” limits the places (combinations of Keys and Values) the object can serve and count as ad impressions. Product: A SASIA system object that is a combination of a Target and information about how much those impressions should cost an Advertiser. Package: A collection of Products. Simulation: The process that SASIA uses to generate the data for calculating Capacity, Availability and Overlap for a specified set of Keys and Values (also known as Targeting or Products). Frequently used interchangeably with Inventory and Forecasting. Inventory: Blanket term for the collection of tools and data used to forecast impression volume for specified sets of Keys and Values. Frequently used interchangeably with Forecasting and Simulation. Forecasting: Blanket term for the act of using the SASIA Inventory forecasting tools. Frequently used interchangeably with Inventory and Simulation. Capacity: The sum of all the impressions served during simulation for a specified set of Keys and Values from all Tiers. Availability: The sum of all the impressions served during simulation for a specified set of Keys and Values from Tiers marked Preemptible. Booked: The sum of impressions served during the simulation from all Tiers from currently Live flights whose targeting matches a specified set of Keys and Values. Overlap: Represents the degree to which the targeting logic from one Target or Product allows impressions to serve to the same combination of keys and values that another Target or Product allows. Delivery Patience: Controls how much patience the system should have in pacing a flight to get onto schedule. "On schedule", in this case, means being at the flight's current "on schedule priority”. For more information, refer to the following SAS Notes: • Inventory Projections • Simulation Forecast Methods • Delivery Patience 11 SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Other brand and product names are trademarks of their respective companies. Copyright © 2015, SAS Institute Inc. All rights reserved.