3 Inspection efficiency
With the reform of the biosecurity sector the Department is moving towards a risk-based approach to Biosecurity regulation. This means, among other things, a balance of targeted allocation of inspection resources towards pathways according to their risk levels and resources towards pathways for the purposes of monitoring their risk. Efficient distribution of limited resources to optimise risk reduction requires scientifically designed approaches. CEBRA produced a range of tools to assist the Department in making more informed resource allocation decisions for more efficient intervention.
Outputs from these projects included tools for identifying risk levels of different individual or combined import pathways and for developing sampling strategies (namely, Continuous Sampling Plans or CSPs) for the inspection of various commodities2,3,4,5,6,7,8,9,10,11 some of which have been operationalised (Box 2, Quote 2). Investigating the pathway for import of medium-risk plant-based stockfeed, CEBRA looked at whether the outcomes of offshore audits could be used to target border inspection efforts and whether results of border inspections could guide audit frequency. Inspection outcomes could not be reliably linked to audit data due to incomplete data capture at the border.12
Box 1: Adoption highlight
Risk Return Projects
The Department adopted and implemented CEBRA tools to introduce risk-based surveillance to the inspection of aircans, sea containers, courier documents, and international shipping vessels. This has significantly reduced the burden of intervention for the Department, but also provides assurance that these pathways contain low biosecurity risk. The Department adopted Continuous Sampling Plan (CSP) methodology to target border inspections for certain low risk plant product pathways under the compliance-based inspection scheme (CBIS). A CSP is a statistical tool that rewards consistently compliant importers with reduced inspection rates. CEBRA developed R code that allows rapid analysis of whether new commodities are suitable for the CBIS. As a result, the Department added seven additional commodities to the scheme. Ongoing work will extend the usability of CSP technology.13 Another CEBRA project has provided the Department with a prototype of a decision support tool designed to assist biosecurity officers with choosing the most suitable CBIS parameters for a given pathway. The tool allows to interactively explore and assess alternative inspection strategies and is currently being tested by the Plant Import Operations branch.
CEBRA solved a challenging problem around passenger profiling which means identifying higher-risk cohorts of passengers that should be preferentially screened upon arrival in Australia. This project identified a component of the passenger intervention, namely the leakage survey, as a potential source of the needed quantitative data. Further, the project advocated the use of an adjustment of the estimates by using a technique called raking, which updates the estimates using the known totals of cohort numbers and channel numbers, delivering improved performance and providing protection against certain kinds of selection bias. The Department has implemented this approach to profile construction for risk management in both the international passengers and international mail pathways.
Quote 2: Brendan Woolcott — DAWE Biosecurity Plant Division (Plant Import Operations)
The work that DAWE Plant Import Operations and CEBRA researchers have done together on the CBIS has produced practical, real-world benefits, not only for the department, but directly for importers. Since CBIS was first implemented in 2013 as a result of our CEBRA collaboration, it has produced $1.8 million in savings for our clients, as well as improving the management of our inspection resources to target higher risk imports.
CSP has been a welcome breakthrough for managing inspection levels on plant import pathways. The department has successfully integrated the CSP methodology into our import systems by allowing us to shift from higher intervention levels without a reduction in assurance. The use of compliance data to automatically adjust intervention levels has been a game changer for Plant Import Operations. We are world leaders in this approach.
Based on CEBRA research, the Department’s prescription for the risk-based management of ballast water by domestic vessel transits has been simplified and strengthened by easier generation of up-to-date risk tables allowing more effective targeting of high-risk transits. Updated risk tables were endorsed by the Consultative Committee on Introduced Marine Pest Emergencies and uploaded onto the Australian Ballast Water Management Information System.14,15 A follow on project compared the use of tide-gauge temperature data with the use of satellite sensed sea surface temperatures. It recommended use of satellite data in simulating life cycles of specific marine pests as part of the Australian ballast water risk assessment. Outputs from the simulation model underpin the development of risk tables and subsequently inform management directions for vessels. In essence, vessels with a high likelihood of transferring marine pests need to manage their ballast water before arrival in the recipient port.16
Over the course of a couple of projects CEBRA developed and trialled a sampling framework for an on-arrival health surveillance system for imported ornamental fish.15,17 The design of a full surveillance system and a software component has been provided to the department, but operational challenges have hampered implementation.
Inspection of imported plant seeds is usually destructive and often requires large sample sizes, making importation of small seed lots (<2500 seeds) unfeasible. Seed importers in New Zealand currently have the option to source bigger seeds lots, but only from countries that are declared free from the regulated diseases listed in the Import Health Standard. CEBRA investigated an alternative testing protocol, considering biosecurity risk for the whole pathway. The study demonstrated the impact of lot size on the leakage rate, which is the proportion of contaminated seeds in the lot that are not detected. One output of the project was the development of a web-based application (an R shiny app) for convenient calculation of sample size based on species and lot size.18
Border intervention activities generally produce large volumes of biosecurity data. CEBRA introduced and tested tools for analysing large amounts of inspection data and achieved operationally realistic predictions of the outcomes of interventions for different pathways, such as international mail and passengers, cargo and vessels.19 In earlier work, CEBRA reviewed data management and analysis processes for quarantine control activities at the border and developed a tiered risk management system, suitable for integration into the operational framework of the Australian Quarantine and Inspection Services (AQIS, these services have since been absorbed into divisions within the Department).20,21
Dealing with smaller data volumes, CEBRA analysed inspection data of consignments of fresh garlic and oranges to test the efficacy of offshore fumigation in reducing contamination rates and phytosanitary risk to acceptable levels. Results from this study showed that offshore fumigation reduced contamination rates of live insects for both products but not to an acceptable level — contamination rates for offshore treated fresh garlic and orange imports were about 9 percent and 34 percent, respectively. The study also highlighted the importance of capturing control-point data electronically and in a consistent manner, which was not the case for phytosanitary certificates,to allow better assessment of the impacts of interventions within individual pathways.22
Using interception, incursion and surveillance records, supplemented by export opinion, CEBRA investigated the impacts of different activities within New Zealand’s biosecurity system on the biosecurity risk of two particular pests. This project provided a rapid prototype of a modelling approach for making reasonable observations about the performance of New Zealand’s biosecurity system but future work will aim to consider collections of pests rather than individual pests.23
Quote 3: Tim Chapman DAWE Biosecurity Animal Division
CEBRA played a vital role in supporting the department’s implementation of risk based intervention to more efficiently and effectively manage the biosecurity risks associated with international passengers, mail and cargo. CEBRA’s analysis highlighted the areas of higher and lower risk which enabled the department to better target its activities at the highest risks in order to get the best biosecurity outcomes. This was a significant change and improvement from past mass screening practices. It provided a foundation for ongoing close collaboration with CEBRA and the application of statistics and detailed risk analysis to identify and target the avenues by which pests and diseases were most likely to enter the country. The result has been better biosecurity for Australia.
Inspection data can also be used to evaluate performance of screening activities. Building on some earlier work, the Risk-Return case study on performance indicators, CEBRA developed an analytical approach to using endpoint data. It is now used to great effect in profiling for and evaluating the outcomes of intervention in the international passengers and mail pathways. Endpoints mark the end of the journey of a unit or cohort of units through the inspection system. Adoption of indicators proposed by CEBRA for the measurement of the performance of passenger and mail intervention has improved the Department’s reporting of operational performance to internal and external stakeholders.24 To improve understanding of inherent biosecurity risks with mail and travellers import pathways CEBRA investigated issues related with the design and execution of existing endpoint surveys. These surveys are required to estimate leakage rates which is the amount of biosecurity risk material crossing the border after intervention. While design of surveys is robust the review identified issues with their execution and provided recommendations for corrective actions.13 Another project confirmed that the existing performance indicators for these pathways are still best practice and recommended to apply them to all import pathways.25
Quote 4: Robert Mudford (DAWE)
I was part of a team collaborating with CEBRA (then ACERA) on developing a profiling methodology and associated performance measures in the international passenger and mail environments. Prior to our collaborative work with CEBRA the idea of a risk-based approach for the clearance of international passengers and mail was only that - an idea. The mathematical and statistical expertise and rigour that CEBRA brought to the department enabled the department to make risk-based intervention a reality, rather than just an idea. CEBRA staff worked seamlessly with departmental staff, combining their mathematical and risk analysis skills with our business knowledge to achieve transformational change in the department. The collaborative development of the risk-based approach substantially reduced the level of unnecessary intervention at international airports and mail centres without increasing the level of biosecurity risk.
The benefit for business and the community was enormous. Lengthy queues for mandatory biosecurity screening and inspection virtually disappeared overnight at Australia’s international airports once the new intervention policy was introduced. Australia Post processing times reduced substantially for low risk categories of mail. Departmental resources could then be targeted at the areas of greatest risk using the mathematical models developed in collaboration with CEBRA. It is testament to the quality of work that CEBRA produced that the profiling methodology and performance measures developed collaboratively with the department are still in use over 10 years later and have been built on by successive teams to increase the efficiency of service delivery, including automation. Personally, I have never been more engaged or achieved more in my career than during the months and years I worked with CEBRA and departmental staff to transform the biosecurity clearance of passengers and mail.
Quote 5: Jose Arias (DAWE)
I manage a team that is responsible for the data analysis to develop the profiles for the air traveller and mail pathways. The methodology developed by CEBRA and the department in 2010/2011 is still the basis of this data analysis in 2019. It is fair to say that it has stood the test of time due to the innovative application of various statistical methods to the limited data that was available to the department. These methods were able to incorporate richer data as they became available to the department. The methodology has been a critical part of the department’s regulatory approach for those pathways since it was first deployed. It continues to allow the department to focus inspection resources where the biosecurity risk is most likely to the highest.
The project reports show the close collaboration between CEBRA and the department to achieve a research outcome that was statistically sound and practically applicable. The department is seeking to continue this collaboration with a project being proposed to revisit the methodology in 2019–20. The proposal seeks to review the methodology using the richer historical data available to the department and also explore new statistical techniques to further refine this risk-based approach.
Compliance with biosecurity protocols is the desired outcome of import regulations and pre-border activities. A scoping study found that incentives for biosecurity compliance might be a feasible approach for speeding up the import inspection process for compliant importers, reducing their regulatory and financial burden.26 Continuing this work, a laboratory economic experiment investigated behavioural responses of stakeholders to changes in biosecurity control strategies. It found that the level of information about inspection rules and targeted feedback on regulatory performance influenced importers’ choice of supplier.27 DAWE adopted these findings by making changes to its operations and providing written feedback reports to importers. A field trial will build on these results and test the effect of reduced inspections on importer behaviour on two plant-product pathways. The suite of ‘Carrots and Sticks’ projects has led to what can be seen as a change in the way how the Department thinks about rules and their implementation (Quote 6). As a consequence, other projects are leveraging from this work, investigating behaviour of stakeholders in the inspection rule setting.
Quote 6: Lois Ransom PSM and Christina Aston — DAWE Biosecurity Plant Division (Plant Import Operations)
Our ongoing collaboration with CEBRA has challenged us to explore new aspects of our approach to risk management. The industry engagement aspects of the ‘Carrots and Sticks’ project demonstrated the value of providing feedback to importers so that they can choose to improve their biosecurity compliance. This is not something we would have proactively pursued without the input of CEBRA experts taking us outside the box.
CEBRA used an inspection experiment to assess the detectability of certain pests in consignments of oranges, providing a framework for further work on improving allocation of inspection resources for fresh produce consignments.28
The Department also inspects plant products for export to a standard specified by the importing country, such as a random sample of 600 units or sampling of 2 percent of the consignment. However, with smaller consignments, CEBRA identified that when the 2 percent option is chosen and <600 units are sampled, Australia’s obligations under the respective International Standard for Phytosanitary Measures are not satisfied. It recommends a random 600 unit sampling, also for mixed consignments. Mixed consignments could be split into lines and sampled at a rate proportional to the number of units per line. However, in contrast to importing plant products, a compliance-based sampling scheme using CSP algorithms is not suitable for sampling of consignments for export.29
References
2. Robinson, A., Hood, G., Cayzer, L. & Scott, M. (2011). AQIS Import Clearance Risk Return - Study A: External container inspection. Australian Centre of Excellence for Risk Analysis, report 1001A Study A.
3. Robinson, A., Cannon, R., Cayzer, L. & Langlands, R. (2009). AQIS Import Clearance Risk Return - Study B: Rural destination inspection. Australian Centre of Excellence for Risk Analysis, report 1001A Study B.
4. Robinson, A., Goldie, S., Gillow, S. & Tognolini, S. (2010). AQIS Import Clearance Risk Return - Study D: Seaports. Australian Centre of Excellence for Risk Analysis, report 1001A Study D.
5. Robinson, A., Karri, R., Clarke, F., Theakston, M. & Scott, M. (2010). AQIS Import Clearance Risk Return - Study E: Unit loading devices. Australian Centre of Excellence for Risk Analysis, report 1001A Study E.
6. Robinson, A., Karri, R., Clarke, F. & Scott, M. (2010). AQIS Import Clearance Risk Return - Study F: Reportable documents. Australian Centre of Excellence for Risk Analysis, report 1001A Study F.
7. Robinson, A., Cannon, R. & Mudford, R. (2011). DAFF Biosecurity Quarantine Operations Risk Return - Study I: Performance indicators. Australian Centre of Excellence for Risk Analysis, report 1001A Study I.
8. Robinson, A., Cannon, R. & Goldie, S. (2012). DAFF Biosecurity Quarantine Operations Risk-Based Approach Overview of Case Studies. Australian Centre of Excellence for Risk Analysis, report 1001B Study H.
9. Robinson, A., Bell, J., Woolcott, B. & Perotti, E. (2012). AQIS Quarantine Operations Risk Return - Imported Plant-Product Pathways. Australian Centre of Excellence for Risk Analysis, report 1001B Study J.
10. Robinson, A. et al. (2014). Plant quarantine inspection and auditing across the biosecurity continuum. Australian Centre of Excellence for Risk Analysis, report 1101C.
11. Arthur, T. et al. (2013). Statistical modelling and risk return improvements for the plant quarantine pathway. Australian Centre of Excellence for Risk Analysis, report 1206F.
12. Robinson, A., Dawes, A., Woolcott, B. & Perotti, E. (2013). Determination of effect of audits on pathway contamination rates for medium-risk stockfeed. Australian Centre of Excellence for Risk Analysis, report 1001B.
13. Chisholm, M., Hoffmann, M., Kuffer, A., Robinson, A. & Callis, S. (2015). Analytical assessment of endpoint surveys. Centre of Excellence for Biosecurity Risk Analysis, report 1301B.
14. Zhao, S., Caley, P., Arthur, T., Robinson, A. & Gregg, M. (2012). Ballast water risk table reconstruction. Australian Centre of Excellence for Risk Analysis, report 1104E.
15. Arthur, T. et al. (2015). Updating the methods for ballast water risk table construction. Centre of Excellence for Biosecurity Risk Analysis, report 1301C.
16. Summerson, R., Zhao, S., Arthur, T., Robinson, A. & McCrudden, R. (2016). Ballast Water Risk Assessment. Exploring new methods for estimating risk: Using satellite sea surface temerature data; Incorporating vessel voyage data. Centre of Excellence for Biosecurity Risk Analysis, report 1501C.
17. Holliday, J. & Robinson, A. (2013). Ornamental Finfish Import Reform Program: A sampling framework and trial for the surveillance program. Australian Centre of Excellence for Risk Analysis, report 1206G ID1.
18. Lane, S., Souza Richards, R., McDonald, C. & Robinson, A. (2018). Sample size calculations for phytosanitary testing of small lots of seed. Centre of Excellence for Biosecurity Risk Analysis, report 1606A.
19. Clarke, S., Robinson, A., Chisholm, M. & Hood, G. (2017). Data Mining - Report for all case studies. Centre of Excellence for Biosecurity Risk Analysis, report 1301A.
20. Robinson, A. et al. (2008). AQIS import clearance review. Australian Centre of Excellence for Risk Analysis, report 0804.
21. Robinson, A., Burgman, M., Langlands, R., Cannon, R. & Clarke, F. (2009). AQIS Import clearance risk framework. Australian Centre of Excellence for Risk Analysis, report 0804A.
22. Lane, S. et al. (2018). Risk-mapping import pathways for risk-return opportunities. Centre of Excellence for Biosecurity Risk Analysis, report 1606C OID5.
23. Robinson, A., Brockerhoff, E. & Ormsby, M. (2018). Scoping the value and performance of interventions across the NZ Biosecurity system. Centre of Excellence for Biosecurity Risk Analysis, report 1606E OID6.
24. Robinson, A., Mudford, R., Quan, K., Sorbello, P. & Chisholm, M. (2013). Adoption of meaningful performance indicators for quarantine inspection performance. Australian Centre of Excellence for Risk Analysis, report 1101D.
25. Hoffmann, M., Robinson, A. & Holliday, J. (2017). Performance Indicators for Border Compliance. Centre of Excellence for Biosecurity Risk Analysis, report 1501F.
26. Rossiter, A. et al. (2016). Incentives for importer choices. Centre of Excellence for Biosecurity Risk Analysis, report 1304C.
27. Rossiter, A., Leibbrandt, A., Wang, B., Woodhams, F. & Hester, S. (2018). Testing compliance-based inspection protocols - Final Report. Centre of Excellence for Biosecurity Risk Analysis, report 1404C.