BRII webinar: Count fish using advanced technologies
Welcome and thank you for joining us today everyone, and we'll be starting this set with about a. Minute. Hi my name is trinity, and i'll be moderating, today's webinar. I'd like to begin by acknowledging, the traditional, custodians. Of the land on which we gathered today. And pay my respects to their elders past and present i extend that respect to aboriginal, and torres strait islander, peoples here today. Some housekeeping, rules, today's webinar, is recorded, and will be available, on the brie website, in coming days. Please keep your microphones, on mute, and if you'd like to ask a question during the webinar, please write this in the q a box at the bottom of the screen. Um that box is different from the general chat, it's on the right hand side of the participants. Button. Um, we'll answer the as many questions as we can at the end of the presentations. If we don't get to your questions, during the webinar, we'll answer these individually. I'd like to welcome brad morton from the department of the industry, science energy and resources. Who will give you an overview of the challenges, and how you can apply. Followed by melanie, olsen and marcus stower, from aims who'll provide, more detail, around their specific, challenge. Brad i'll now i'll now hand over control to you. Okay thanks hi my name is bradley morton and i'm the program manager of brie. I've worked in the bree, program for the last three and a half years, and managed the last two rounds of brie. Okay so what is bree, bree enables, australian, government, agencies, to tap into leading-edge, thinking, and seeks innovative, solutions, from startups, and smes. Small to medium enterprises. To challenges, identified, in public policy, and service delivery. So how does, this work. For this round. The program covers, five environmental. Challenges, for smes. Small to media enterprises. Can develop, solutions. These challenges, were proposed, by australian, government agencies. And the successful, challenges, were recommended, by innovation, science australia, and approved by the minister. Smes. With a great idea are invited to apply. And address the challenges, through the development of a new solution. We do understand the new solution can be a modification. Of existing, technology. The best applications, will receive, upfront, funding for a feasibility, study and the winners will receive, grants of up to a hundred thousand dollars to test and document, their ideas over three months. The most successful, ideas, and feasibility, studies are then. Eligible, to apply, for a further grant of up to one million dollars to develop a prototype. Or proof of concept, over the following 18 months. Once a solution has been developed, the sme, has the advantage, of an agency, considering, buying, your, solution, or selling your solution to the industry sector. Smes, are also encouraged, to sell these new solutions, worldwide, as you retain the intellectual, property rights to your solution. Please note some challenges, will have the government agencies, intended, end user, others will not. For example the previous round i tree. Delivered, the world, first, child safety information, sharing platform between state and territory governments, from a large, government contract, afterbury. Other challenges, have addressed a gap in the market, for example mars and jacob associates, waterflow, app. Was designed to give farmers and irrigators, free up-to-date, water trading information.
The App offered both free and paid content, in. In this instant, the government, agency, was one of many potential, customers, using the premium, subscription, service. While you're solving the specific, challenge for the government agency you also need to ensure you develop a solution in a way that allows you to capture more than one customer. And if possible, more than one market. This is why we say breed gives businesses, the chance to work with government, create a novel solution. That could be launched. Globally. Funding and cash flow. Breed provides, smes, with critical, early stage financial, support for the development, of new to market innovations. First part is the feasibility. Stage. Successful. Applicants, are eligible to receive up to a hundred thousand dollars to undertake a feasibility. Study for three months. With two million dollars in this stage, it works out to around four hundred thousand dollars available, for each challenge. The second stage the proof of concept. Grants. Successful, feasibility, study participants, will then be eligible, to apply for up to one million dollars to develop, a proof of concept or prototype. With 10 million dollars in total, funding for this stage it works out to around two. Million dollars available for each challenge. Working with agencies. Bree is not about giving you the dollars and then leaving you to develop, your solution in isolation. The program has inbuilt monthly, catch-ups. Where we, encourage. You and your company particularly during the feasibility, stage, to have, discussions. Questions and answers conversations. And workshops, the more the better. The program allows, the grantee, to do a presentation, at the end of the feasibility. Stage and the proof of concept stage so you can show and explain. Your solution, to the agencies, and relevant. Key stakeholders. The department of industry will manage, the administrative, side of the grant. The challenger agencies will provide, information. Context, and expertise, around the challenge to be solved. And you and your company are responsible, for delivering, the solution. That solves the challenge. And then market. And launch, into the future. Maximize, your opportunity, for success. Bree provides, many opportunities. Such as, significant. Funding to entrepreneurs. Smes, and startups to test explore, and develop commercially. Viable. Innovative, technologies. This is not match funding, if selected, you can access, up to 1.1, million. Dollars. Bree also gives you the chance to gain a significant, reference, customer, and track record leading to facilitating.
A Path to market and a new opportunity, for future investment, and growth, for example, likely theory, a round one grantee, developed convalence. A web-based, consultation. Platform, that enables, better engagement. And consultation, with communities, and departments. As part of their research and work they ended up getting a contract, with department, of prime minister and cabinet. To do a piece of work on the aps review, about two-thirds, of the way through their proof of concept. Free also gives you the opportunity, to gain and improve, knowledge experience, and develop relationships, with australian, government clients. It allows you to have the retention, of ip, rights for you to commercialize. Products. Concepts. Domestically. And internationally. Pre also, provides, the opportunity, to explore. And exploit, spin-off, products. Another round one grantee, atamos, project, has led the business to being involved with the innovative, vector, control consortium. To develop technologies, to combat malaria, and other deadly mosquito-borne. Diseases. Which is being led by the department of foreign affairs and trade. Brie also provides, the potential, to access, other government assistance, programs, such as, accelerating, commercialization. Okay let's have a look at eligibility. To be eligible for the feasibility, study and the proof of concept, grants. Smes, must, have an abn. And have a turnover, of less than 20 million dollars. Per year over the last three years. There is an exception for organizations, who are controlled by australian, universities. Or public, sector research organizations. See the guidelines. New, newly established, companies, are welcome to apply. Joint applications, will be accepted, provided the lead applicant, is both the primary, project, proponent. And meets all other criteria. And individuals, and partnerships, may also be considered, eligible, if they agree to form a corporation, that meets. Other, eligibility, requirements, before, signing the grant agreement. Those that are not eligible, are income, tax-exempt. Companies. Government, agencies. Federal state and local, and government business, enterprises. And trusts will not be eligible, although a corporation. Acting as a corporate trustee. May apply, on behalf of a. Trust. Merit criteria, this is what each, of the applications, will be assessed on. A, suggestion, is, get to the point, not too much jargon, as the assessment committee may have a lot of these to read through. Merit criterium, one extent that your proposed, solution, meets the challenge.
How Is the proposed, solution, different to what is already in the market. How is it new how is it innovative, and how does it solve the challenge. Merit criterion, 2, market, opportunity. Of your proposed solution. The market need for the proposed, solution, within government. The future commence. Commercial, potential. Of the solution in domestic, and international, markets. For example you could go into how. Would it be commercialized. Include the market opportunities. Flexibility. And scalability. Of the solution. Could it be used in other industries, expanded, into other markets, etc. Merit criteria, three. Capacity. Capability, and resources, to deliver the project. This is where you can include your track record, in managing, similar projects, and access to the personnel. With the right skills and experience. Your access. To infrastructure. Equipment, technology, and intellectual, property. You can include, a sound, project, plan, to manage, and monitor the project and its risks. And also your project budget. So it's a good place to focus on your past achievements, similar, successful, projects. Key personnel. And their experience, and. Qualifications. All right how to apply. The application, form, is located, on business.gov. Dot au portal. The address is on the screen. Some things to note about the application. Form are you can start it and save it and come back to it. Some sections have character limits. Do not leave, it until the final few hours to complete. Be mindful of relevant time zone differences. The application, will automatically, close at the scheduled, time of 5 pm. Australian eastern standard time on thursday, the 10th of september. And any applications, still open and not submitted at that time will not be able to be submitted, and we do not accept, late applications. If you have a great idea, and meet the eligibility. Criteria. We strongly, encourage you to apply. This is a screenshot, of the, business.gov.edu. Page. I would recommend you checking out that page. It has information, and templates that you'll need to apply such as the grant opportunity, guidelines. And templates, required for the application, such as the financial. Turnover. Template. And also letters of support template. Thank you. Thanks brad hopefully that answers everyone's questions, about how they can apply for a challenge, please keep your questions, coming, through, i'll now hand over to melanie from aims, to provide more detail about their challenge. Thanks janine hi everyone welcome and thanks for coming. So marcus and i will presenting. Detail regarding our brie challenge, today which is counting, fish with advanced, technologies. Next. Slide. It's a bit about ames, so ames is the australian, institute of marine science, and. Where australia's, tropical marine research, agency. We conduct, marine monitoring, and research to support the sustainable, growth and environmental, management, of australia's, tropical marine australia. Estate, so that's all the way from the great barrier reef. Right across northern and northwestern. Australian, marine waters. Next slide. So let's talk a little bit about fish monitoring. Australia's, fish population. Supports. Many local communities, and industries. Right across australia, but particularly, in the north. Fish diversity. Size and abundance, are indicators, of the health and sustainability. Of marine ecosystems. Including, coral reefs. And ames collects, fish information, to understand, marine ecosystems. The impacts of external, effects, and change over time. Bruvs, or the baited remote, underwater, video. Station, is a common, fish monitoring, tool used by ames and its. Partners. Next slide. So what is bravs. Bros, consists, of a metal frame holding an inexpensive. Camera, or two when in stereo configuration. And those cameras are pointing, at a bait arm. This bait arm has a mesh bag full of pilchers, that attract sea life and lights can be added for deeper water surveys, too. So brubs, are normally deployed. From aims in shallow, or sub 50 meter reef areas. And it's common to see between. 10 or 70, species. In a brubs video but depends on the biodiversity. At the coral reef location. Typically. Bruvs, are deployed, in fleets of eight or more, and the video is recorded, for about 60 minutes. So each station, itself has a rope and surface marker for retrieval.
And Then, when you get back to base the video footage is analyzed. Where fish are identified. By specialists. Marine experts, collect information, on species, and individuals. Community, structure. And fish size. And the data is used to inform decision makers, on the status of coastal fisheries. The effectiveness. Of marine. Protected, areas, and to empower, local communities, to monitor and manage your own sea country. So all bruvs. As a device, can be deployed at scale and we do. The current data processing, methodology. Can't, it's really time, intensive, and it's a manual process, so we've got an expert marine researcher, sitting there watching and manually, analyzing. Each video. Next slide. So the challenge. The challenge is to provide. Is to find, an automated. Scalable. Solution. That solves the intensive, data processing, problem, and is easy to use. So fish experts should be able to use the tool without needing, a software engineering, degree. And if we can solve the challenge, routine fish monitoring, would be scalable. Across tropical, australia. Next slide. We know that there are advanced, imaging. Filtering. Machine learning classification. Object tracking technologies, out there. So is it possible to count fish using advanced technologies, hence the challenge. And can we integrate these challenge these technologies. Into a reliable, tool that can be used at scale by bronze users, across australia, and that's why we submitted this to brie. So next slide please. We're just going to talk through some, important. Solution. Considerations. So there are some features that we need to um. Specify, just to make sure that the brothers, unit does collect the data that we need at the right, uh well it meets the scientific, need. So the solution needs to accurately, capture scientific, info, there's three elements to that one's fish biodiversity. And feel free to ask questions on this later because we've got marcus on board he's an expert. Marine biologist, you can answer any detail. That, that you might have any questions. But we need to identify, the fish taxa, a daily, down to species, level for a set target, species, or taxer in the video. We also look at species, relative, abundance. So maximum number of taxa, seen in each video so that's. Basically you should find the frame, in the video with the maximum, number of fish for each target taxa and record that number, and then that frame, should be recorded, to enable the user to verify, the species, abundance, count. And the final. Scientific, info element is the fish biomass. So, we measure fish length to estimate, biomass. Presently, it's achieved, manually, using stereo, photogrammetry. And, other approaches, that could provide accurate, measurements, might also be considered, in a future solution so we're trying to capture that whole workflow. As, keep the box as open as we can for this challenge. The solution also needs to be intuitive. And easy to use. And it also needs to be cost effective. So the solution will address a large bottleneck. So it's got to be able to scale efficiently, and cost effectively, and ideally it'll just open up the entire northern. Shores forest. In terms of brubs. So, the challenge is open to innovation, from a whole systems approach. Are there any new data collection, or imaging, technologies, that may make the data better suited. To automated. Or semi-automated. Analysis. Or. Are there semi or automated, analysis, technologies, that could be returned, routinely, applied, in a way that quickly, creates, scientific. Validated, data sets. Now ideally the solution would be adaptive, to evolving, technologies, as the future. Improved, imaging systems, models and techniques. Develop, as well because it really is one of those evolving, fields at the moment. Why is this an important issue to solve. So understanding, fish abundance, size and biodiversity. Is really, important. Researchers. Would better understand. Reef ecosystems. And the impacts of external, effects. Decision, makers, would be better equipped to manage coastal, fisheries, and reef ecosystems. And finally, being able to scale bruvs, across, remote. Regional, australia, means the local communities, and traditional, owners would be empowered, to understand, and manage their own sea country. So it's more than just research, more than just ames. So what broader, uses could the solution, solve. Really it's up to your imagination, in a lot of aspects because it is one of those key fundamental, technologies, that could be a game change industry-wide. But from from this perspective, i could see the solution, creating new opportunities, for fish monitoring, on other underwater, platforms. So, the towed and remotely operated, video systems. Autonomous, underwater, vehicles, diver operated, video systems and underwater, live stream cameras. So used by a variety of marine industries across australia. And an efficient.
Fish Monitoring, analysis, solution, could be applied, to these, other video collection, techniques, which would translate, to more information, for researchers, users and decision makers. So see what i mean by it could really scale. There are a variety, of citizen science initiatives, across australia such as iron the eye on the reef, by, gabrumpa. That could potentially, adapt, an efficient brubs analysis, workflow to collecting fish information, as well as coral. And the ability to identify. Pests or endangered, species, present in collected video, imagery, could greatly enhance the awareness. Of the populations, of these species of interest. How can aims, help successful, applicants, during this challenge. Amos is really keen to see this challenge be solved so if you govern an agency out there and do you want a partner. Hit us up because we want to solve this once for australia, not just for, our little um, little agency monitoring, the tropical north but. We're just keen to see this problem solved you'll see it as a capability, gap in australia. So to help applicants. We have bruvs, videos, for testing ideas. Most have manual identifications. And fish counts, that may or may not be useful for machine learning training it depends, on, on the algorithm, and the tack to take. And we also have extensive, experience, in broad monitoring, and access to the national, and international. Broads community. For solutions, looking at machine learning approaches. We do have the auspicious, data set which i've linked to here just google it, it was developed, as part of the australian, research, data commons data discovery. Program, with the university, of western australia, and curtin university. And, as i said before we do have other classification. Um, sorry other annotations. That may be of use but this one was put together to try and, help. You guys and other partners to try and solve the problem. So, thanks very much for your interest, in our challenge. And. We hope to see your ideas, on, how to count fish using, advanced technologies. Thanks both for your presentations. We've received quite a few questions. So i'll hand, this one over to, both brie, and to you.
Melanie. This is a. Question specific to the ames challenge are all commercialization. Paths valid for submission, more specifically. Is it okay to propose a commercialization. Path outside of just developing, an ecological, monitoring, service. I can start with this one if you like. Brad. From my perspective. I really see. The whole brew process, about trying to make australian, industries, stronger, and more competitive, internationally, and really grow that in growing capability. So. If there was an innovator, out there who could solve our problem and then extend that beyond to other commercial, paths. I don't see how, about why, we wouldn't support that. Yep i can agree there basically that's what we sort of have, as part of the merit criteria, so the merit criteria, one is to solve the challenge. Numeric criteria, too is how do you commercialize, it and grow your business, yep agree. Thanks for that um we've had some questions, about data sampling. Is there an opportunity, to access, a small amount of aims broads data to develop ai models for image recognition. And some analysis, has also asked can you provide sample data under various light conditions, for the feasibility. Study. And also one can you make some video, files available for training data. Ai, and ml training. All right i'll start with that one feel free to jump in marcus. But in terms of the brav. Data, set we've got all sorts of videos, and all sorts of. Quality. Light conditions, turbidity, you name it, and our intent is to, to put up, an archive, online. And just how we'll link that we're hoping we can link it through, the breeze site, if everyone wants to have access to it. But it will just be a series of our videos packaged, up that does have a bit of that diversity. And in terms of the machine learning classification. Elements, um, we do have that data set that i mentioned on the last slide. If there's anything in addition to that for example the annotations, that go along with the videos.
That We'll put in our archive even though they might not be in the right format. I'm sure we can work to put those up as well. Okay thanks for that, um then there are some more specific, questions about, bravs are the current bravs a step into the current graphs of standard, design. And what light do bruvs operate in natural light artificial, or both. Uh yeah i can answer that one um. Yeah there's certain, parameters, that are certainly, standard. In the world of graphs. So whilst, the actual frame design, does, vary, slightly, between, some different operators. We have some, basic, principles. Of, camera separation. And height above the seabed, and that kind of thing that are that are held. Constant. And, and that's true for. A whole range of different, people around australia, that are using graphs. Um, for various, purposes. So there's a certain amount of inertia, there if you if someone needed or wanted to change the design, of the gear. That that's a complicating. Factor because we've tried to develop it as a standard, method. The, the lights. Typically, we use natural, light if there's sufficient, available, natural, light. So down to, maybe 70 or 80 meters of, depth we would just use natural, light which retrieves. Reasonable, footage. For visual analysis, anyway. And then beyond that. Ames uses, a custom, made. White led. Spotlight. To illuminate, the field of view. I, can't tell you the specifications. Off hand but it just provides, a general, white illumination. Of the field of view. There are some other. Bravs, people around the place using. Blue and red led, lights for illumination. But ames has found that, white is the optimal, one, for image quality. Thanks for that marcus. Um we've got quite a few technical, questions, about. Um. The video. And bruvs, so i'll just keep, keep working through those. Are you currently collecting, video on an sd, card. And a human is watching the entire video in the lab while counting, fish how frequent, frequently, do you do this. Yeah i'll answer that one as well um. So yeah the videos are recorded, to an sd card which we then download, in the field, onto hard drives. And then the current, analysis. Process. Is. To for a. Human, to re-watch, the video. Um. We do have the option of playing, back it faster than real time. It's. Really paced by the the speed at which the person can watch the video and record, what they're seeing. And. Through the analysis. Of the entire, video, we're looking to identify, every species, of fish that occurs. And also. Keep track of, the number of individuals, of each of those species, and we look for a, metric, called the max n, which is the maximum. Number. Visible, in a single, field of view throughout, that one hour video, for each. Species. It's certainly, fairly laborious, so, watching. Um. You know dozens, of videos. End to end. So we have some questions about the metrics, and data sets what are the most important metrics for the system one person asked and also. We have a system that uses machine learning fish recognition.
And E, uh edna, collection can you help share more. Of what you'd really like our platform is capable of doing a bunch of different data sets what is the most wanted data set. You need as a scientist. So. Our, fundamental, metrics. For, for broad's, work. Are. Fish species. Richness, or a current so we want to have a list of all the species, that occur. On a given deployment. And we also want a measure, of each species, abundance. And um. Given that. By their nature. A camera system, uh. Sort of records, fish coming and going throughout the hour we can't know exactly, how many fish there are in. In a particular, deployment. We use a, surrogate, of abundance, called max end which i mentioned before, is that. The peak, number, of each species, that occurs, throughout the hour. This might you might have already answered this but what is the most important, action after analyzing, the video content. Uh. I'm not sure i understand, the question like we really have to turn the video into numbers, for a statistician. To be able to take it from there and do analyses. On it, but obviously, the, data and important process, for us is. Ensuring, that the data is archived, and stored. In a database. So, fundamental. To the to the harvest, of numbers, from each video, is. To put that in a centralized. Location. So that it's accessible. And, preserved. In a secure, way. Is there additional, information, recorded, with the fish logs for example. Weather. We do. Measure. Some water quality, parameters. Independently, of the rub so we use ctds. And. Other, scientific, instruments, to. To measure. Locality. Parameters. Um. Currently, there aren't any other sensors, on the brubs, units themselves, apart from the video, cameras, and. The lights, as well. And do the sample videos, contain, the labels, of the fish species. Not, in our. In our current, process. We don't actually, label. The, individual. Fish, throughout a video. But. We've recognized, that that's a necessity. For a machine, learning type approach, so we have got some data sets that have been, appropriately. Annotated. Where. Boxes, have been drawn, around the fish and and, those, boxes, have been identified, with a species, name. And that there's another question in there that is similar to what you just said so regarding, the data set can you share more about how images have been annotated, for example bounding boxes. Or instant, masks, uh poly, polygons. Maybe mel can, can answer that a bit better. Based on what's what's been done to date, um. I wasn't involved in that annotating. For the machine learning exercises. So far. Yeah i might take that one on.
Notice. Okay sure. I have a feeling bounding boxes were drawn around a whole heap of fish but that's just an instinct, i have, yeah, they definitely, were i just don't know if they went to a mask level after that and if that's, what was, in that data set online. How large or how and how many annotated, images, is the existing, data set do you know. I want to say 84, 000. Images, in there but i don't know, what the breakdown in terms of biodiversity. And the number of fish. Is. Off the top of my head. Okay. Um. Uh. What's the current workflow, for loading and reporting, on the videos. So, i can answer that one um. Typically, we'll come back from a field, trip. With all our videos, on a hard drive, as mp4. Files. They're. Typically, recorded. In. 1080. Pixel, high definition. At 30 frames a, second. We'll. Come back into the lab, and. Load those videos, into. Either, of two, annotation. Software, packages, that we use. One is an. Aims. In-house. Microsoft. Access. Database. Front end. Which enables, you to view and, view a video. And at the same time. Record, what you're seeing into an access. Database. The other software, package, that serves the same purpose, is a commercially, available, one. Called, event measure, produced by. Cgis. Proprietary, limited. And and that. Essentially, does the same thing where you launch. Where you load the video, in. And. As you're watching the video, at, a. Selected, playback, speed. Pausing, and, recording, as you go you, you. Manually, enter the species, names, and, manually, count the number of fish. From there. The data, is, already, in the aims database, if we've done it through the aims. Access, database, it feeds straight into. The oracle, database. Or if it's, been done through the event measure package. We then have to load. The emobs, files, which are the. Resultant, files, in event measure, into our database, and that's where the data is stored. And then from there people extract, the data, and do all their analyses, and that kind of thing. Are there often laser points included, in the imagery. And what is the estimated. Error in species, id and manual process. And what is the current gold. Standard. Uh there's, not we don't currently, use lasers. Um. The. Photogrammetry. Is, is based on a stereo. Method. So there's, no need for scaling, lasers, with that system the whole, camera, assembly, is calibrated. And then. Using. Custom, software, which is, the event measure package. By cgis. You can actually do a. Snout. To tail fork measurement, just by, clicking, on the, on the points on your screen. I guess the scaling, lasers, are an alternative, technique, but. I guess the the orientation. Of the fish and all that sort of thing. Comes into play as important, factors. What was the second part of that. Question. Um. I've um put it into the answered folder which has 45, so sorry i can't retreat. I'm saving it forever. Um. That's okay we can move on. Can you please clarify, what you mean by counting biomass. Do you aim to count total weight of fish on the screen by breed. Or are you aiming for a solution, that may extrapolate. Calculation. To a certain larger, area eg. Several, square kilometers. That may include several bruvs. So. We're really interested, in in knowing how long the fish are that's a sort of a standard metric, in fish biology. Is is measuring, the. Snout, to. Tail fork length. And then if we've got a series of those measures, for different species, we can from there. Use that measurement. Um. To. Determine, the approximate. Biomass. Of the fish so um. So we're really just interested, in, knowing how big the individuals. Are that are visiting, these. Stations, when they're on the, seabed. I think melanie was going to go and um, answer the bounding box question. Yeah i probably was like i couldn't answer before but i've got the the link up now so. It, was, bounding boxes so the data set itself is that oz fish linked one.
It's, 80, 000 labeled, crops of fish extracted, from videos. So over 500, species, 200, genre and 70 families. And there's 45, 000 bounding box, annotations. Official, no fish across, 1800, frames. Hopefully that answers the question if there's any other detail you want to know about that data set, the link, um is the metadata. For it so it should be. Uh well there for you to have a look at. Um, we've asked if, some people have been asking if we can add the data set link to the chat, um so just put that in the notes and if we could um put that in the chat and send it out to all of the um. Attendees, that would be great. Uh. So, we've got some specific, bruvs, questions, um, could you give you an idea, of the cost of deploying a single graph system this would help us determine, the baseline, cost to aim for with hardware. And also how many bruvs, are deployed, per vessel, at one time, and how far apart, per brass, eg, one vessel ten brands deployed, over one square kilometer, is that feasible. Okay, so. Um. A brass, an individual, brubs, unit. Is, probably, worth in the vicinity, of. Two to three thousand, dollars. Just for um. Materials. Cost and the cost of the cameras. Um. But that. That hardware, cost is. Is exceeded. Um, by probably an order of magnitude. And the cost of um being out there with the staff required, to deploy, them so. Um, there's a significant. Labor cost involved, in using these things. I guess it's not, really, a requirement, of the challengers. Here to have their own grubs, hardware. It's more about the software. Interface. That the challenge, relates. But anyway. In, in terms of deploying, bravs that you're really limited by the size of your vessel. We would typically. Deploy. Eight to ten robs at a time and, they're. Typically, spaced. Around, half a kilometer, apart. Um. But it sort of varies. According to the purpose, that we're and the location, that we're deploying, them. People. Operating, out of smaller vessels, might use less than you know four to six at a time or something like that.
If The, data processing, was quicker, and cheaper, when we would probably, deploy, more. Great. Um. Are you currently, using, r or python, or other, for stats. Analysis. Certainly, r is in. Very common use, at aims. Among the statistical. People. Let's say we can, provide that data you want how do you plan on using this data to promote, 30 by 30, under, sdg, 14. I'm not sure i know those acronyms. No same, not sure. Jeremy if you could just clarify, that would be great i've left your question open and i'll just move on to something else, have you tried applying, ai already, what challenges, did you face, or is this the first attempt at applying. Ai, conf, computer, vision. Um, and there's also another, ai, question. Down below while you answer that i'll find the other one. I i can answer that one. I guess if you like. So it's not just names but there's a variety of people across australia that have, tried a couple of different, ai, techniques so the off-the-shelf, ai. Um. I know at ames we've had a couple of initiatives. Going. But i don't think anyone solved it yet so there's summit, i've seen, um, a couple of university. Sectors, as well occasionally you'll see a media release where people are counting fish with um bruvs or other, systems, we're trying to, but we're really up to that operational, system because our browser's operational, so we've got to try and get something that we can just, get in the field and scale and that's what i've not seen, today. Someone said there seem to be commercially, available, ai systems that can already do this have you looked at some of them which is similar to the above question. I i think we've tried. Virtually everything that's, off the shelf and the problem that we've got is. Um, just the pure, biodiversity. That we have in our our um. Our braves deployment, so on the reef you've got as i mentioned before, the 10 to. 70 different species, and with, with um brows with our footage. It varies in turbidity, and lighting conditions, so. It's not repetitive, enough for those off-the-shelf. Systems, we've just got, so much breadth for geographical. Coverage. And the biodiversity. Within that, do you want to add to that marcus or that cover it. Only that, i i guess my. Casual observation. Of what's been done so far is that this machine, learning approach is only as good as its training, data. And. A big, bottleneck. Perhaps, is. Is the harvest, of that data and the input into the process, so i think. There's probably, potential. For a, system that learns as it goes if you like. And then that that might be a way of, addressing, these. Issues that melanie, mentioned, of varying, turbidity. And varying, light conditions, and all of that sort of thing. You know it's. It's a massive, imposition. For us to have to. Try and, in advance, provide, training, data.
For A system. So it, sort of needs to learn as it goes, so to speak. Okay. Um, in terms of the stereo, camera setup can you ensure, at least one sample video, set includes the video from both cameras. Yeah i don't think that would be a problem so just to video file say marcus. Yep that's easy to. Supply. And also another very technical question, do we have access to the fish taxer. Uh. Certainly. We can supply, the identification. Information. To go with a video. I think that's what that one's getting at. And how many images, in the data set and how many different species, of fish are represented. In the data. Set. This must be the oz fish one yes that was 80 000, images. And. Up to 500. Species. Okay. Thanks for that is there any interest, in analyzing, the images, at the site. I'll answer that if you do have an embedded, system solution that could give you a quick look or a. Um i process, it on the fly, that's just one less step in the chain which would make it more efficient, so i think that'd be awesome i just don't know if it's possible. Regarding, the annotated, data set was the medium, of the data, video from abruvz. And was this video, was this video, or individual, images. I think it was still, images, taken from the video. Yeah that's my understanding, is that the. Original. 1080. Pixel. Video, files, were split into individual, frames. And that and the individual, frames used as images, is that right now. That's my. Understanding. Okay. What is the boundary, between, what is in the feasibility. Study. I.e a statement, what of what is possible, versus, the delivery, of the actual, actual solution. Um, it was unclear to me from the application, how much solution, is needed in the first component. Right here from the program team i can help you answer that so, the feasibility, study is. As per the same merit criteria, as basically, telling us what your idea, is. How you can address that, challenge. And. Basically we want to know, and and there's budget questions et cetera so that's all around 100 000, all around the first stage. But definitely, include. You know. What you can see into the future how you can do it in the proof of concept, etc, in there as well so. But focus it on the feasibility, stage. In the first instance, but if you've got plans for proof of concept, in the next stages definitely include that as, some, additional information, for sure.
Back To some technical, questions, is your focus, on coral reefs or are you also interested in tropical, open oceans. For example, pelagic. Species. I would hope that any solution, would be appropriate, to both. All of our. Grants, deployments. Are at the seabed. And, i guess, at one level that introduces, a level of complexity. That, uh that is. Higher than a pelagic, environment, where you might have just a blank. Blue background. But, you know hopefully, this solution, will be appropriate, to both. Can i ask how valid the flow of currents, us, to specific, species, that follow these patterns do you already have this data that could be applied to other methods, of the observation, for monitoring, numbers of fish stock. Um. I guess currents, are certainly an important. Ecological. Parameter, but they're not something that we typically, measure, um. In conjunction, with our brush deployments. Oceanographic. Data is sometimes, sort of brought in separately. For the analysis, for the big picture stuff. It comes down to scaling. A bit a bit too so we want bruvs to be, scalable, and quite low cost as soon as you start getting. Um. Current profilers, and that type of thing the cost just jumps right. Up. What tools are currently used to capture the outputs, is there a proprietary. Sw. To play the video and capture annotations. Notes and findings. Uh, yet there sure is, there's, one. Proprietary, software, package, called event measure i've mentioned. Already which is, commonly, used. Throughout australia, by and internationally, by people doing props. And then there's also, the. Ames bruvs, database. Which. Uh has been set up to. Annotate. The videos, but not do the. Stereo, measurement, aspect, of the videos we still rely on event measure for the photogrammetry. Aspect. And the aims broads database. Is. It's not proprietary. Software, it's just microsoft. Access. Interface. Based on provided, challenge description, for the fish counting. Um. Challenge for aims. We understand that you expect to receive statistical. Data, for example fish type quantity, size biomass. Biomass, estimate. About, fish filmed at, the bruvs, what do you want to do with the data how to use it do you want us to export it to your own database, via your own interface. Or would you like us to design our own database, to store all processed, information, so you could connect to it using, api, or other tools. Okay so this time if you like marcus. I think um an api. Of course is future proofing and enabling. A, bigger market perhaps for the product. I think in terms of aims itself, of course we'd always like to have backward computer, compatibility. With our own oracle. Database. But completely, understand, that if we have to put a step in there to get whatever the new um. Workflow, is. Feeding into the database, that would be part of it. Let's say we can provide that data you want how do you plan on using this data to promote 30 by 30 under, sdg. 14.. Right, when you said sdg, before i was thinking sd cards cause sustainability. Development, goal 14.. Sorry. Um. Yeah so, this is that previous question as, well. The way aims. Aims as an agency, itself our mission is to. Ensure that, the growth, in, our region is sustainable. So that does track up to the un sustainable. Development, goals. For the perspective, of this challenge though, it really comes down to that fish monitoring data set and being able to collect that at scale across our region so. We can get those, that data, to inform decision, makers, on those, bigger picture goals. And at the moment we don't have that so, i would see. The bruv's data if we can, start getting that scalable. All of a sudden, you can get that time series change, right up and down the coast in australia which could help put us in a position to help informing. Whether the 30 by 30 is realistic, or whether. Um. Where to put it for example and start informing decision makers but at the moment, it's a little bit. Tough from the aims perspective, because we can't scale. Okay. What, would be more beneficial, for you between, automating. Counting, or measurements. Are they equally important, or how would you rate their importance. Versus the opportunity. To improve your workflow. Can i jump in and say that. Most of our time is spent, on the, accounting, and identification. Aspect, of this so. If. If it was simply, a matter of, being delivered. All the, accounts, and ids. And, a couple of mouse clicks to complete the picture with the measurement that would be fine for us. Does each station, have the same settings. Yep. We have standard. Settings, on our cameras. I think you might have answered this but the person has asked it again so i'll just, ask it again um to make sure that um no answers have been lost what is the.
Estimated. Error rate in current manual process, i. What is the gold standard. As of now in species, id. So i guess it's only. Two there's two constraints, here one is your. Quality of your video analyzer. And we'd hope that, that their accuracy. Would be, extremely. High, sort of you know 99. Or something like that. But there is a, constraint. In this technique. In that. Your identification. Is only within this constraints. Of, the video, quality you're looking at so. There's always a proportion, of fish that are not going to be identifiable. To species. Level. Um. To put a number on that i'm, sorry i can't but i guess our, benchmark. For measuring the performance, of any of these systems, would be, against that of a human being, so. We would hope that something. Automated. Would be able to ultimately, go close. To the, capability, of a human being for identifying. Things. Yeah i think just to add to that marcus in terms of validating. A. New process, or a new automated technique i think that's how we would do it is you'd, have a video there that. The computer, or machine analyzes, and then have. Uh, same. Standard. Marine experts. Doing the analysis, and then comparing. But you do have that issue where. Is, the human wrong or is the computer wrong so it's it's something, that the entire autonomous, industry is struggling with at the moment. Well stereo, video samples. Be included in the sample, set and, the calibration. And is the calibration, information, available, to us or, is proprietary. To cgis. Now we can certainly, supply. A cal file, to go with the rig that it was recorded, on. I guess that. The. Issue, there is to be able to utilize, that cal file, as probably. Requires. Access. To the cgis. Software. Could we please, current hardware, specs. Uh yeah i, guess. I, could probably best. Provide a. Technical, drawing, or something like that but um. But, the systems, are based on. Two gopros. Hero4s. Or. Or more recent, um. Five degrees. Of convergence. On each camera, and. Um, separated, by 750. Millimeters. How do humans. Handle, duplicates, for example, if you. Left and then enters again. Yeah so we. We're, we're only looking for. Max n which is. The maximum, number, of each species. That can be seen in, a single, frame. Throughout that one hour video. So it totally, ignores, revisits. Uh, or, you know it doesn't take into account revisits. Of fish it's actually, a. Conservative. Estimate, of the true abundance. Abroad's, always stationary. Um or are there cases that they are moved during filming. Houses are always. Stationary. But having said that sometimes, the background, can be moving. For example if there's current or something like that. You might have some motion, in the. Background. What is the overall number of expected, taxa and species. It depends. Entirely, on the location. And a. Low diversity, deployment, might have. Five to ten species. Uh high diversity, deployment, might have. 70, or 80 or more species. Just depends on the location. Uh how often how, far offshore are the bruv systems. Typically, deployed, can they be left out under water for extended periods of time for example days weeks or months. It's typically, it, would just say it's just limited, on. The battery life of the camera so we do a 60 minute. Film, and then retrieve. The units. But in terms of um, if they could be down there with the. Camera that stops and not filming anymore. Marcus would say they they could be left for a couple of hours will we do the retrieval, between them all would that be right. Uh, yeah one one hour is, um. The standard. That's been adopted, by most broad operators, throughout australia. Uh that's a legacy, of, one hour of videotapes.
Being Common. Back, back in the day of videotapes. But it's also a practical amount of time to leave them down. But having said that there's. Like you say that battery life's really the only limitation. To longer deployments, and we have done all day deployments, with. Bigger batteries. And in terms of how far offshore, we deploy them. We're. Typically, interested. In. Continental, shelf waters, down to, around 150. Meters. With most of our current work. And most of it is shallower than that sort of. Less than 50, i guess would, account for most of our, deployments. Beyond, looking for the data and automating, the process of identification. Are there challenges, to be solved, in the location, of where to deploy the bravs, is it, very clear, when and where to measure the fish. We've got statisticians. That. Sort of, guide us in how we deploy, these things. I guess we're always, interested. To see. New, solutions. Or new possibilities. For how these things get deployed, but. Um, yeah, often their. Sampling, designs, have been. Prescribed. By a statistician. Given, the question, behind the study. What if we could offer a better data collection, mechanism, than braves, hardware, where we could include, anglers, themselves, bruv seems very isolated. When data touch points across a fishery. Needs to be vast, to get an appropriate, data set. We're always, open to the possibility. Of. Sort of citizen, science and that kind of thing i guess. The difficulty. With angling, and rubs. Being, sort of. Tied up together, is, the biases, that come with, where people want to fish and all that kind of thing so. I guess. The answer that is that the the question. Behind the study has to fit the technique, that's, that's employed. And. You know we're typically, looking for a fishery, independent, method, for surveying, fish. Um. You've kind of answered this but i'll ask it again just in case there's, another. Angle. To it what is the typical, filming, area of ruffs. What are the ranges, of the, analyze, zone, for example depth width and length. I haven't got the, numbers, on the top of my head but. Just as a rough indication. That, the bait arm which is sort of often the focal, point of where the fish action is, is one and a half meters in front of the frame. And we will typically, measure, fish out to eight meters. Beyond that the precision, of the stereo, measurement, technique. Tends to decline. So we. Don't, measure beyond eight meters distant from the frame. And. The vision, laterally. Is, determined, by the field of view of the cameras.
Off The top of my head i think it might be around the 80, degree. Mark the, field of view of, a gopro, underwater. Does the brush collect audio, as well when recording. No. And how much video, footage, measured, in hours did ames capture vibe robs in 2019. Ah. Uh. Let's, let's see. I i guess. It would be hundreds, of hours. Um. Just trying to think. We might have even gone over the thousand, hours last year i think. So it's in that ballpark, sort of high. High, hundreds, to a thousand. Uh veranda, is a typical. Amount of sampling. What's the current error rate and what do you see expectation. I think that tracks to, marcus's, question mark has answered before, about. At the moment we have a human. Doing the, analysis. And if there's. There's just some that you just can't, look at because um, the quality is just not there or they're too blood. But in terms of an actual error rate. We don't have that comparison, so it's really, the human we're just expecting, is this best that we can get some say around 90. Well. You can't measure and i suppose is the way we're looking at the movement. We do um, quality assurance, exercises. Where we cross reference, one person's, work with another. So. We sort of have a bit of a sense of how accurate a human is. And uh and that. Ten, tends to be. Um, you know, close to a hundred percent, within the limitations. Of the technique. I guess one thing i should throw in there in terms of accuracy, though is that. Currently, the, analysis. Process, for these videos, is. Purely, manual, so it requires, a human. Expert. Specialist. To watch the video from beginning, to end. And. If we could. Improve. The, workflow. By automating. It. So that a human is still required. But. It significantly. Lessens. Our amount of intervention, then that's a big gain for us so there's. The scope, here. For less than, perfect. Accuracy. Um, to still, you know improve the whole process, i think and, still have. The intervention, of a specialist, to bring it up to that hundred percent that we're aiming for. Thanks for that is the solution, expected, to measure the countered, fish. As well. Um. And does bruvs, provide indicative, scaling metrics, to allow. Determination. Of where an observed fish sits in the field of reference than another similar question. When measuring, each length do you select, the largest, fish per specie in a video or do you measure the length of fish. So. We don't use, a, physical. Scaling. Method, because. We use. Stereo, photogrammetry. To measure the fish and. With that technique, the. Field of view has sort of, been. Mapped, i guess if you like so we can actually just measure things through mouse clicks. What was the other. Aspect, of that question, or, whether we measure the largest fish no we. We typically, take the video, to that, max end point for each species. And we measure. Every individual. Of that species, at that point of the video, so we're trying to get a representative. Sample. Across, the population. Of those. Fish. How open are you to. The addition, of to additional, physical hardware to the bruv system, a minor mechanical, modifications. To the brass system tolerable. Contain detailed mechanical, drawings of the front. System. I think we're, trying to promote. A really open solution, approach to this so for example, if, there's a mind modification. That could really make the analysis, process, so much simpler. That was still cost efficient and met that overall objective, of brothers and making sure it's scalable. I think we would support that. There's certainly. Potential, to add payload, to the frames. I guess, when the modification. Gets to the point that it. Affects, the standardized. Design, of the, unit that's, that's when we have to weigh up the benefit. Of that modification. Against, its impact, on our, standardized. Method that's been used, for. 15, plus years. What are your budget limits for fixed cost expenses, and ongoing expenses. Of a proposed. Solution. I find that hard to answer because we don't know if it's going to have the hardware, element, as well as the software element but to give you, an idea. At the moment, as you mark is how if you had a brubs video. How long would there be an expert, marine researcher, sitting there to watch one video and do the analysis. Uh. I guess there's probably, a, couple of hours. To analyze. An average video. So. So say. Three human hours just to, just to analyze. And, and deliver the data. Um. That's it's abuse that is a baseline. For. Something if you had something that was scalable.
Well That's what we're paying now, like across the industry, so. That's what i would use as a baseline, i guess. And some. Question that then ties in with that is there a time requirement, for the processing, videos for example, hours days weeks. Oh it's always the sooner the better but um. We we tend to set our deadlines. Based. On sort of a known, timeline. Processing. So the quicker we can deliver the better. What's the what's an average, max, in. So a max n is that, um. Peak number of individuals, of each species. Uh, in one frame throughout the one-hour, video, and an average, max n. I guess is the max end value, for a species. Averaged, across a number of brass, deployments. Okay great, so um we can't take any more questions and we'll just work our way through, the current questions, now. Um. Are you, this might have been answered but are you looking for a solution that would be hosted on your infrastructure. Or that will be hosted, and managed by us and you work with it via api, or via a web interface. I think it really depends, on the implementation. With it so as long as we will always have a copy of the data. Locally, or with your names records. And then however. Whatever if it's cloud-based, or on-premise, or however the tool. Does that transition. As long as it's. Efficient and cost effective. We would then store that data, that process data sets back in. In line with, the archives, act and records management. Practices, that we have to comply, with. So in terms of cloud-based, or premise, on-premise, i think it. Probably depends, on on the solution or not, hard. Bent, either way. What's a typical, internet, speed upload, download, and connection, type. Satellite, nbn. 345g. Available, at the place where. Um a marine biologist, would do video analysis. At ames it's, we're on the rna background for backbone, so, it's, really fast ambient equivalent. However. If you did want to scale this you'd probably be thinking. More along the lines of someone being able to upload a video. Set across regional australia. With, satellite. Satellite. High bandwidth systems are getting cheaper but it's typically. On land, at a research, station, so, you're looking at. Nbn, or adsl, or one of those. Right. Do we refer to the classification. Taxonomy, from the fishers, of australia. Fishers of australia don't need to iu. That's certainly, a, quality. Online, resource. It's. There's. Occasional. Uh discrepancies. Between, that and, and, taxonomic. Acceptance, standards, but. Taxonomy, is always, a bit fluid, anyway, so there's always debate, but certainly that that's a, good. Online, resource. For the. Good standard of taxonomy, that we would employ. Is classification. To family level acceptable, or do you really need to be at the species. Level. Oh no we're operating, at species. Level. So, i guess. Family level would only be an, intermediate. Solution. For us. And in some ways i think it's. It's not a, as easy a solution, as as a species, level when we're dealing with. Image recognition. Because. At the family, level. The characters, that are distinguishing. Families, are not always. Visual, characters. Uh there are a lot of empty frames in the video. Where there are no fish in view what percentage, of the 60 minute video would you say this would. Be. Depends, on the location. Um. In in some cases, there might be, uh, say 30. Of the video that's empty frames. Uh in a coral reef environment. 100. Of the frames will have fish in them. It just depends, on the location. But there's definitely, a time saving, to be had. In being able to eliminate, any frame that has no, fish. How do you visualize, your results, at the end of the day. Through. Various statistical. Analyses. I guess. You know. It serves a whole range of different purposes. From. Modeling, of, species. Distributions. To. Plots of fish abundance. Comparisons. Between, sites of fish abundance. All those kind of things so it's. The visualization. Is, really a statistical. Thing. And just one final question. Can we get a sample of real video, and annotation. During feasibility, study. No matter what format it is. Yeah absolutely. Preferably even before the feasibility. Study. Hit so we'll work on that, as a priority, to get something up that we can link perhaps on the breeze site or, share with interested people. Okay. Um. So that's it for today. Thank you everyone. For attending, if you'd like to know more information, or download, fact sheets please go to business.gov. Dot dot au forward slash brie. Um if we weren't able to answer any questions, they'll be on the brave, website along with the recorded, webinar.
Otherwise Please send through any queries debris, at. Industry.gov.eu. Have a good day and stay safe. Excellent thank you guys. Thanks. Everyone.
2020-08-29 00:43
