Flow data processing

Outline of the video

• Let’s look at flow data processing first. Go to the processing tab and choose the flow data queue. Assign it to yourself and start the work.
• This category is where you will have most of your data.
• First show you how you can start to work on a file and then discuss in which format the data needs to be.
• The flow data processing template has 10 columns.
• Period, this is the time reference
  • whole year if start date to end date really cover the entire year
  • Q1, for quarters is also possible. But actually we won’t need data that detailed.
• Start date and end date, the format that we need is YYYY-MM-DD
• Material/product name Ideally in English. (The original name can be in the comments column) You can choose the same one of the EMP code or one that is quite close. If you have several detailed materials, you can use the segment column for that.
• Material/product name, EMP
  • Materials catalog
  • Note: Remember that the overview of EMP codes for each of the 12 and 11 materials of each sector plus their wastes exists for your reference too.**
• Quantity, just the number
• Unit
  • Unit should match the “unit symbol” from the List of units. Be sure to use “kg” instead of Kilogram or “t” for tonnes.
  • Capitalisation matters, so be sure to use correct spelling.
• Reference space
  • Here is where you add the location name that this data is about. For example, your city name “Porto” or your NUTS3 name. Remember that we made an overview of those for you.
  • (Although it will be unlikely in your case, it is possible to add a file with several reference spaces. For example with data on your city AND NUTS3 or country. This only works if you have a shapefile that also has both of them in one file, to attribute the correct spaces.)
• Comments
  • Optional field and can remain empty.
  • You could add a URL to more info, the original language name of the material, or simply a comment that you changed the unit.
• Segments: Here you can become more specific about the material, see examples in the template spreadsheet.
• Finally, please add “_processed” or “_final” to the previously used file name, save it and add it during the processing.
• M11:50, uploading a processed file and going through the rest of the processing
• M14:25, how to fill in the data quality information

Example 1 - Extracted rock and gravel

Outline of the video

• A real case example about flows
• Amount of extracted rock and gravel in Mikkeli
• Different material names and units
• from the info in one of the columns we can see that the data is from 2019, so we add that
• find the correct EMP code from the material catalogue
• M5:00, Converting the unit from volume to mass for stone, and sand and gravel. You can use the conversion table from the Economy-wide material flow accounts handbook, in this case, table 14.
• you don’t have to add the totals, the system does that automatically

Example 2 - Crops in Porto

Outline of the video

• Focusing on an example in the biomass sector in Porto
• example: Hectares of agricultural area used in Porto by Composition of the agricultural surface area used. This example is a bit too broad to showcase.
• other example: Hectares of permanent crops in Porto by Type
• Clean up the file: unmerge cells, remove columns that don’t have any information. Note: If you do these types of actions, you can refer in the processing section of data quality that there is more data on other types of crops, but that there are no production values for them or that they were 0 in that year.
• Conversion from hectare to tonnes, for which we need the yield in Porto or in Portugal in general
• M8:44, FAO data section
• M10:18, convert the CSV data into columns
• FAO recorded this yield data for almonds and grapes in hg/ha, hectograms per hectare (100 grams)
• The mass in kg can then be derived from the hectare values that exist for Porto.

Note: Actually the wrong code was used for almonds. Although you can also make almond oil, they are NOT oil-bearing crops, but instead considered nuts, see here. If you are unsure about what a material is considered as and grouped under, then you can always check the FAO grouping.