Few people like the thought of "Big Brother" watching them at work. But despite numerous protests, fishermen in many regions of the world have had to get used to the fact that their on-board activities are observed and recorded around the clock. On large and in the meantime also increasingly also smaller fishing vessels electronic monitoring systems record everything that happens on deck when the crew is at sea. Cameras connected to motion sensors and GPS systems document which fish species are caught, in what sizes and quantities, whether the fishermen comply with authorised catch quotas, and how they deal with discards. These records enable the regulatory authorities to check whether all the pertinent rules and regulations have been complied with or whether any illegal actions have occurred. For example, unauthorised transhipments, transfer of fish to other vessels on the high seas. These actions, which are otherwise difficult to control, often serve to conceal the origin of catches and are also linked to human rights violations such as slave labour on board ships. And another important aspect: the evaluation of these data also serves scientific progress, enabling fisheries researchers to calculate fishing pressure more accurately and thereby improve management strategies.
Although the vast majority of fishermen adhere to all regulations and laws the authorities’ "distrust" is not completely unfounded. Despite considerable improvements around 20 per cent of the world's fishing catches – or put more simply every fifth fish – still come from illegal, unreported and unregulated (IUU) fishing. IUU fishing costs the world economy up to 23 billion US dollars a year, poses a threat to the sustainability of fishing in our oceans, and contributes to human rights violations. It is not completely new for fishermen to have someone looking over their shoulders while they work since a lot of fishing vessels regularly have fisheries inspectors on board who are employed to monitor fishing processes and catches. This is expensive and also not very popular because every extra person on board takes up additional space and also disrupts normal working procedures. The use of electronic surveillance technology can now, to a certain extent, replace the inspector’s expert eyes with the lens of a camera. This is not the biggest difference, however. What has changed is the extent of surveillance. While even under the US-American ‘At Sea Monitoring Programme’ only 15 per cent of fishing trips are accompanied by human inspectors – in tuna fishing in the Pacific Ocean between Indonesia and Hawaii the proportion is said to be only two per cent, i.e. every fiftieth vessel – the level of camera usage is much higher. On larger fishing vessels it has now reached almost 100 per cent in industrialized countries.
Intelligent technologies can take over tiresome routines
However, the almost complete monitoring of fisheries now presents the authorities with a new and even greater problem: the camera recordings only serve their purpose if they are carefully evaluated. This is the only way for inspectorates to get a reliable picture of fishing practices and detect illegal activities. That means that someone in the control authorities has to watch the vast amount of videos that have been made throughout the duration of the fishing trips. It is a monotonous, tiring and uncreative activity that ties up workers. And it is not cheap. For that reason some control authorities only carry out random checks of the records and then compare their findings with the fishermen's catch logbooks on a "trust basis". This, however, thwarts efforts to control fisheries effectively and has led to new approaches for dealing with the situation. Machine learning and artificial intelligence are now to help convert the huge flood of images into more usable "big data".
The computers employed here have special software which – similar to image recognition software for the identification of human faces on social media sites – can evaluate the video images on the basis of extensive image databases so that the fish caught can be assigned automatically to specific species. The digital tools have to learn which visual characteristics are typical of which fish species, for example what a cod looks like and what distinguishes herring from mackerel. In spite of artificial intelligence, machine vision and deep learning this is quite a challenge as the fish often struggle when they are taken on board and so can go past the camera lens in very different positions. The more photos the database contains, the more reliable the software becomes. This problem has to be resolved if the automated evaluation of video images is to deliver useful results. And recognizing the fish species is only the first step, because the software will then also have to measure the length of each individual fish and convert it into their weight, because these two values are the decisive basis for scientists' calculations and thus for fisheries management. As we know, it is almost impossible to manage something that cannot be measured.
Video review companies around the world are now investing in machine learning, and it will probably only be a matter of time before a commercially viable product becomes available. In Australia, The Nature Conservancy's FishFace Project is photographing thousands of fish, recording lengths and weights, and collecting further information besides to build up the software. The Swedish company Refind Technologies places its cameras in special light boxes to standardize the general conditions of the images, in other words to guarantee constant light conditions and constant distances between the camera lens and the fish that pass it by. Each fish photographed is then assigned its species name – manually for the time being. In New England (USA) an online competition was even launched to speed up the development of a reliable fish image analysis system. The basis for this is an open source software which is to be "taught" exact counting, species identification, and fish size measurement. Whoever does this best will receive prize money of 50,000 US dollars. The results of the competition exceeded all expectations and the winners achieved almost 100 per cent accuracy for counting the fishes and 75 per cent for identification. This was sufficient proof of the fact that automated image evaluation can work in principle.
Acceptance of innovative technologies steadily increasing
The algorithms of electronic monitoring could perhaps even help to resolve the latent distrust between fishermen and fisheries scientists. Researchers often doubt the data noted down in the fishermen's log books, and the fishermen in turn sometimes have little confidence in scientific analyses. However, GPS and video data provide credible evidence that the fish were caught in the designated area in accordance with the quota. The scientists can thus obtain dependable data and the fishermen can convincingly show their customers where their fish comes from and which methods were used to catch it. This creates confidence, facilitates marketing, and is a useful starting point for traceability concepts. Ecotrust Canada is implementing ThisFish, the first traceability programme of its kind. Consumers can enter a code online and get information about who caught their fish, how and where, and sometimes even watch an online clip.
Considerable progress has been made in electronic monitoring of marine fishing through artificial intelligence, machine learning and digital image recognition. On some vessels data is now transmitted in real time via the Internet, even from distant fishing grounds. Because the connection is sometimes unstable the technology has an online-offline mode. If the internet connection fails, the data is stored offline and only uploaded to the cloud when the connection is restored. Japan is already trying to exploit these fascinating technical possibilities for real-time fleet management. The intelligent business platform Smart Fishing Operations will be used to guide ships to rewarding fishing grounds. This should save time and fuel, increase the efficiency of fishing, and also conserve fish stocks by reducing by-catches and avoiding local overfishing.
The U.S. National Ocean and Atmospheric Administration uses artificial intelligence not only to identify caught fish species but also to monitor dead zones and currents in the sea and measure pollution. In 2015, an intelligent system for the electronic monitoring of fish stocks was launched. A device positioned on the seabed has a computer and sonar system that is activated as soon as it detects fish movements in the surrounding area. The number of fishes at that particular location can be estimated from the reflected pings of the sonar. Some of the devices are to be placed in the Arctic in remote regions which are hardly accessible in winter without expensive icebreakers. In this way, fish stocks and their movements could also be tracked below the ice.
China is also using big data and artificial intelligence to improve national fisheries and aquaculture management. To this end, a new subgroup within the China Fisheries Association (CFA) was founded: the China Intelligent Fisheries Association. The CIFA will bring together data specialists, fishing companies and government officials to structure and organize data collection for management objectives.
AquaCloud helps salmon farmers control and combat salmon lice
In the fishing and aquaculture country Norway the Seafood Innovation Cluster, an industry-funded organisation, seeks to promote the growth of the fish industry through the use of data sciences and intelligent technologies. High on their list of priorities is the fight against the salmon louse plague which has become a serious threat to Norway's wild salmon population and its important salmon farming industry. According to cautious estimates the direct cost of managing and controlling these ectoparasites is at least $600 million a year, and some experts even reckon it to be higher at $1 billion. In addition, salmon lice retard the further growth of salmon production because if a salmon farmer cannot prove that he has the lice problem under control he is not permitted to expand his business. To solve this chronic problem the Seafood Innovation Cluster has joined forces with IBM to develop AquaCloud, a platform that collects data from salmon farms across the country and uses intelligent machine learning techniques to analyse and predict the spread of lice along the Norwegian coast so that farmers can take appropriate defensive measures to prevent infection of their salmon populations. AquaCloud is a predictive analytics platform that automatically collects data and warns of possible salmon lice invasions. It is now apparently possible to predict salmon lice infestation with an accuracy of 70 per cent (a level which is expected to rise) which significantly accelerates the farmers' ability to act.
In all sectors of the economy that are for the most part dependent on natural resources it is important to maintain a balance between usage, protection and preservation of that resource, and a company’s success will be closely linked to its ability to achieve this. Fisheries and aquaculture have some catching up to do in this respect and so they need to learn and implement this lesson faster than other sectors because global demand for fish and seafood is constantly growing. Ultimately, the social acceptance and very existence of the global fish industry will also depend on its ability to increase yields without compromising natural resources through overfishing, disease or environmental damage.
High-tech companies discover the fish industry as new business segment
Japan's fish farmers use advanced technologies such as artificial intelligence, cloud computing and drones to reduce production costs and improve operational processes. They are supported in these efforts by the country's technology and telecommunications companies, e.g. Sharp, KDDI and NEC. These companies have recognized that fish farming is a booming industry that offers lucrative growth opportunities, and they hope to turn high-tech solutions for aquaculture into a new business segment. For example, at a Hiramasa farm in Miyazaki Prefecture, NEC tested an intelligent image evaluation technology that measures the length and width of individual fish and, based on these results, automatically calculates their weights. This enables the farmers to adjust feed quantities and feeding times more precisely to the actual needs of the fishes. In the meantime this technology has been extended to include tuna, and the developers believe that it will also be suitable for other fish species.
The Japanese telecommunications company KDDI Corporation, which mainly specializes in corporate networks, is developing very similar technologies that take water temperatures into account during feeding, for example. Intelligent sensors are used here. Since falling fees in the telecommunications sector are weakening revenue KDDI is looking for application fields for products such as sensors. Although the profits that can be achieved in aquaculture are still small the industry has enormous growth potential. Companies from Taiwan have already expressed interest in the KDDI technology which is based on the ultra-fast 5G mobile radio standard.
The next step will be to test the sensor-based technology on oyster farms in Hiroshima Prefecture, the most important oyster production region in Japan. While sensors on buoys and rafts measure the water temperatures and salt concentrations in the area, drones search for swarming oyster larvae and observe water current patterns. Intelligent computer programs then use the data collected to determine which areas would be most suitable for positioning collectors for the settlement of oyster larvae.
All over the world attempts are underway to use data and automation processes to increase efficiency within the fish industry, to manage it more sustainably, to reduce labour costs in aquaculture and fishing, to avoid environmental pollution, to prevent overfishing, and to ward off diseases in fish farms. Big data and artificial intelligence have become important drivers of economic development in the fish industry. They make aquaculture and fishing more predictable and reduce the risks associated with them. Almost a decade after the introduction of electronic monitoring on fishing vessels in the US and the EU there is sufficient evidence that the principle works. Combined with catch quotas, fishing effort limitations and extensive documentation of catches these controls have made fishing more sustainable. The collected data play a crucial role towards monitoring and managing fish stocks more effectively.