Can we scrape Australians’ personal information from public websites?
- Published
- Law stated as at
Key points
- Copying (or scraping) personal information from a public website is a ‘collection’ under the Privacy Act. A company must need the information for a current purpose, gather it by lawful and fair means, and be able to justify collecting it from sources other than the people themselves.
- Sensitive information includes membership of a trade union or of a professional or trade association, health information and criminal records. Collecting it requires each person’s consent, and no exception applies merely because asking thousands of people is impracticable.
- Removing names afterwards does not make an unlawful scrape lawful. The Office of the Australian Information Commissioner (‘OAIC’) treats gathering identifiable information with the intention of de-identifying it as a collection of personal information, and de-identification as a further use of that information.
A company that wants Australians’ names, trades and phone numbers in bulk rarely asks the people concerned for them. Instead, it uses a scraper, software that automatically copies information from web pages, to build a dataset from directories and social media. If the company is bound by the Privacy Act, the dataset is lawful only if the scraping complied with that Act’s collection rules. A company that breaks those rules can be required to stop collecting and destroy what it has gathered.1 This article explains those rules in more depth, including when consent is needed, what de-identification can and cannot achieve, and what a buyer of a scraped dataset should check.
Is scraping a public website a collection of personal information?
Scraping personal information from a public website is a collection of that information. A company collects personal information when it gathers it for inclusion in a record, such as a document or an electronic device, or in a publication available to the public.2 The OAIC, in its guidelines on APP 3, the collection principle, interprets the word ‘collects’ broadly. On that interpretation, collection covers information obtained from any source, including public internet pages; by any means, including data scraping and web crawling; and even information held for only milliseconds.3 The other half of the question depends on what is scraped: a name, photo or review is personal information if the person is identified or reasonably identifiable from it.4
The collection rules bind Australian companies, and also overseas companies that carry on business here, unless they are exempt small businesses, generally those whose annual turnover is $3 million or less.5 A separate article on overseas companies explains what it means to carry on business in Australia. It includes the tribunal reasoning that copying images from servers in Australia can amount to carrying on business here.
What must a company show before it scrapes personal information?
A company that scrapes personal information must be able to show three things: it needs the information, it cannot reasonably collect it from the people themselves, and its methods are lawful and fair.
First, the information must be reasonably necessary for one of the company’s functions or activities.6 The OAIC applies this test objectively: information that is merely helpful or convenient does not meet it, nor, in most cases, does information gathered in case it is needed later.7 The OAIC’s guidance for developers of generative AI models (the ‘AI guidance’) asks them to specify a current purpose rather than collect information for an undefined future product.8
Second, the company must collect the information from the person concerned unless doing so is unreasonable or impracticable.9 The guidelines ask whether collecting directly would impose an excessive burden in all the circumstances, and say that inconvenience, time or cost alone is not enough to justify collecting indirectly.10 Even so, contacting thousands of people may impose such a burden.
Third, the means of collection must be lawful and fair.11 The OAIC gives hacking, trespass and breaching a court order as examples of unlawful collection, but says that a breach of contract is not unlawful in this sense.12 A website’s terms of use are instead relevant to fairness, because the guidelines count such terms among the factors that shape what people reasonably expect.13 The guidelines also describe covert collection as usually unfair and say that information on the public internet may not be collected and used however a company likes.14 The AI guidance goes further, treating the building of a dataset by scraping as generally a covert method of collection.15
When has scraping public information been found unfair?
Fairness turns on the facts, as two tribunal cases illustrate. Clearview AI built a database of faces from the public internet using a web crawler, so that law enforcement agencies could match photos against it.16 In reasons given in May 2023, the Administrative Appeals Tribunal was not satisfied that the crawling was unfair, because people had put their images online without restricting access.17 Those reasons never became a formal decision. The OAIC says that Clearview then withdrew from the review in August 2023, before the Tribunal had decided what orders, if any, to make.18
By contrast, Court Data Australia was found to have collected personal information unfairly. That business, operated by JFA (Aust) Pty Ltd, copied entries from a Western Australian courts portal’s daily lists into a database that customers could search by name. Court Data Australia argued that anyone listed in court would expect to appear in such a database. In May 2025, the Administrative Review Tribunal disagreed and affirmed the Australian Information Commissioner’s finding of unfair collection.19 The Tribunal said that automated collection was not unfair in itself, although it made the collection process less discriminating.20
The Tribunal considered the circumstances of the person who complained, whose criminal charges had appeared in the lists and were later dismissed. That person had no say in the listing, was unaware of the collection, and had no reason to expect a short-lived court list to become a permanent commercial database that invited adverse conclusions.21
The portal’s conditions of use also supported that conclusion: they prohibited commercial reuse without the court’s permission. Court Data Australia argued that it had not breached those conditions, pointing out that the portal had never complained. The Tribunal said that this missed the point: given the conditions, a reasonable person would not expect such reuse, so it did not need to decide whether the conditions had in fact been breached. The Tribunal also rejected the idea that whatever is posted online is ‘a free for all’.22
In practice, a company planning a scrape should consider each source and ask whether the people chose to publish the information, what purpose the website serves, and what its terms of use say.
Can a company scrape sensitive information if it cannot ask for consent?
A company cannot scrape sensitive information without each person’s consent unless a listed exception applies. No exception applies merely because obtaining consent is difficult.23 Sensitive information includes health information, criminal records, ethnic origin, religious beliefs, sexual orientation, political opinions, and membership of a union or of a professional or trade association. It also includes biometric information that will be used to identify people automatically.24
The exceptions cover narrow situations, such as collection required or authorised by law; a serious threat to a person’s life, health or safety; suspected unlawful activity or serious misconduct; and legal claims.25 The closest exception for research covers only health information collected for purposes such as public health research, and only where, among other conditions, de-identified data would not be sufficient and obtaining consent is impracticable.26 So the OAIC expects AI developers who cannot obtain consent to delete sensitive information from a dataset. It also says that sensitive information collected accidentally will generally need to be destroyed or deleted.27
Does de-identifying scraped data afterwards help?
De-identifying scraped data afterwards does not make an unlawful scrape lawful. According to the AI guidance, gathering identifiable information with the intention of de-identifying it is still a collection of personal information. De-identification is then itself a use for a secondary purpose, that is, a purpose other than the original purpose of collection.28 The order matters. If a scrape collected sensitive information without consent, or was unfair, the breach has already occurred, and later redaction does not undo it.
De-identification can still reduce privacy risks once the data is held. The AI guidance says that a developer that cannot show that its collection and use comply with the Privacy Act may need to delete sensitive information before using the dataset. It may also need to delete or de-identify other personal information. Data is de-identified, and so is no longer personal information, only if nobody is reasonably identifiable from it. A record without a name can still fail that test, for example, where a rare job title and a small town point to one person.29 Once a company no longer needs the information for any purpose the Privacy Act permits, it must take reasonable steps to destroy the information or de-identify it.30
What must a company tell the people whose information it scrapes?
A company that scrapes personal information must take reasonable steps to make people aware of the collection, the circumstances in which it occurs, and its purposes. It must do so at or before the time of collection, or as soon as practicable afterwards.31 Where it is impracticable to notify each person individually, the AI guidance suggests publishing and promoting a general notice on the company’s website, ideally a reasonable time before any training takes place.32
We are buying a dataset someone else scraped. Is that any safer?
Buying a scraped dataset does not shift the privacy risk to the seller, because the buyer also collects the personal information.33 For AI developers obtaining a dataset from a third party, the OAIC lists eight checks:
- whether the data contains personal or sensitive information;
- whether the data can be narrowed to what is reasonably necessary;
- how the information was first collected, including its sources and any notice given to the people concerned;
- whether the third party will vouch for where the data came from and how it was gathered;
- whether the developer’s own means of collection are lawful and fair;
- whether the people themselves could be asked to provide the data instead;
- whether there is valid consent for any sensitive information, which must be deleted if there is none and no exception applies; and
- what changes are needed to the developer’s privacy policy and collection notices.34
In practice, the seller’s warranties about where the data came from may give the buyer a claim against the seller if those warranties prove false. They do not, however, relieve the buyer of its own obligations under the Privacy Act.
How would the rules apply to a scraped directory of tradespeople?
The example below is hypothetical. It traces a proposed scraping exercise from the question of who is bound by the Privacy Act through to what the company must do once it holds the data.
Suppose a Singapore software company with turnover of about $2 million and no Australian office plans to sell Australian builders a searchable list of subcontractors. From 1 December 2026, it wants to scrape 10,000 public profiles of electricians, plumbers and carpenters from online business directories and social media.
- Does the Privacy Act apply? Selling its product to Australian builders indicates that the company may be carrying on business in Australia, which would bring it under the Privacy Act. Its turnover of about $2 million is below the $3 million threshold for the small business exemption. However, selling the list discloses the tradespeople’s personal information for a benefit and without their consent. That takes the company outside the exemption, regardless of its turnover.35
- What should the company collect? The product needs names, trades, business contact details and service areas. The AI guidance advises AI developers to exclude sources such as public social media profiles.36 In practice, the company should follow that advice and avoid scraping social media profiles as well.
- What must it leave out? Many profiles state that the tradesperson is a member of a trade association or a trade union. Membership of these organisations is sensitive information, so the software must exclude it. Any such information collected inadvertently must be deleted.37
- Is the scrape fair? The tradespeople published their business details for the purpose of finding customers, and the planned use is closely related to that purpose. In practice, the company should also avoid scraping any directory whose terms prohibit commercial reuse.
- What does it need to tell the tradespeople? Before 1 December 2026, the company should publish a notice on its website giving its name and contact details. The notice should explain that the company collects tradespeople’s details from directories, what information it collects and why, and that builders who use its list can see those details. It should also explain that the company’s privacy policy sets out how a tradesperson can access and correct the information and make a complaint.38
- What should it do after the scrape? The company must take reasonable steps to ensure that the directory is accurate, up to date and complete, and to protect it from misuse and unauthorised access.39 Its privacy policy must also explain how it collects and holds the information.40
Questions
- The website’s terms allow scraping. Is that enough?
- A website’s permission does not, by itself, satisfy the Privacy Act, because the collection rules protect the people the information is about, not the website owner. The scrape must still be reasonably necessary and fair, and sensitive information still requires the consent of each person it concerns.
- Does the Privacy Act reach a company that scrapes from outside Australia?
- The Privacy Act can bind a company that scrapes from overseas if it carries on business in Australia. In 2023, the Administrative Appeals Tribunal reasoned that the US company Clearview AI did so by collecting images from servers located in Australia, although the Tribunal made no formal decision before Clearview withdrew its application.
- How long can we keep scraped personal information?
- A company may keep scraped personal information only for as long as it needs the information for a purpose the Privacy Act permits. After that, it must take reasonable steps to destroy the information or de-identify it, unless an Australian law or an order of a court or tribunal requires the company to retain it.
This article is general information about Australian law as at the date shown above. It is not legal advice and does not take account of your circumstances. Reading it does not create a lawyer–client relationship with SQ Legal. Obtain advice on your own matter before acting. See the terms of use.
Related
- Can we use Australian customers’ data to train an AI model?
- Does the Australian Privacy Act apply to a company based outside Australia?
- Are tracking pixels on our website a privacy problem in Australia?
More on data and privacy, or contact SQ Legal about a matter.
Anna Zheng, ‘Can we scrape Australians’ personal information from public websites?’, SQ Legal (Web Page, 16 November 2026) <https://sqlegal.com.au/insights/web-scraping-personal-information-australia/>.
You may quote short extracts with attribution and a link to this page. To republish the article in full, write to [email protected]Email[email protected]Open in your default email appGmailOutlook (personal)Outlook (work or school)Or copy the address:[email protected].
Footnotes
-
Privacy Act 1988 (Cth) s 52(1)(b), (1A) (‘Privacy Act’); see, eg, Court Data Australia and Office of the Australian Information Commissioner [2025] ARTA 876, [3] (‘Court Data Australia’). ↩
-
Privacy Act (n 1) s 6(1) (definitions of ‘collects’, ‘generally available publication’ and ‘record’). ↩
-
Office of the Australian Information Commissioner, ‘Chapter 3: APP 3 Collection of Solicited Personal Information’ in Australian Privacy Principles Guidelines (Guidelines, 13 May 2026) [3.6], [3.8], [3.11] (‘APP 3 Guidelines’). ↩
-
Privacy Act (n 1) s 6(1) (definition of ‘personal information’). ↩
-
Ibid ss 5B(1A), (3), 6C(1), 6D(1); Clearview AI Inc and Australian Information Commissioner [2023] AATA 1069, [197], [200] (‘Clearview Review’). ↩
-
Privacy Act (n 1) sch 1 cl 3.2. ↩
-
APP 3 Guidelines (n 3) [3.25], [3.27]. ↩
-
Office of the Australian Information Commissioner, Guidance on Privacy and Developing and Training Generative AI Models (Guidance, 21 October 2024) (‘AI Guidance’). ↩
-
Privacy Act (n 1) sch 1 cl 3.6. ↩
-
APP 3 Guidelines (n 3) [3.90]. ↩
-
Privacy Act (n 1) sch 1 cl 3.5. ↩
-
APP 3 Guidelines (n 3) [3.81]–[3.82]. ↩
-
Ibid [3.87]. ↩
-
Ibid. ↩
-
AI Guidance (n 8). ↩
-
Clearview Review (n 5) [1]. ↩
-
Ibid [132]. ↩
-
Ibid [201]; Office of the Australian Information Commissioner, ‘Statement on Clearview AI’ (Media Release, 21 August 2024). ↩
-
Court Data Australia (n 1) [1]–[2], [20], [42]. ↩
-
Ibid [32]–[33]. ↩
-
Ibid [10], [29], [33]–[37]. ↩
-
Ibid [11], [19], [40]–[41]. ↩
-
Privacy Act (n 1) sch 1 cls 3.3–3.4. ↩
-
Ibid s 6(1) (definition of ‘sensitive information’). ↩
-
Ibid s 16A(1) items 1–2, 4, sch 1 cl 3.4. ↩
-
Ibid s 16B(2). ↩
-
AI Guidance (n 8). ↩
-
Ibid. ↩
-
Ibid; Privacy Act (n 1) s 6(1) (definitions of ‘de-identified’ and ‘personal information’). ↩
-
Privacy Act (n 1) sch 1 cl 11.2. ↩
-
Ibid sch 1 cls 5.1, 5.2(b), (d). ↩
-
AI Guidance (n 8). ↩
-
APP 3 Guidelines (n 3) [3.6], [3.10]–[3.11]. ↩
-
AI Guidance (n 8). ↩
-
Privacy Act (n 1) ss 5B(3), 6D(1), (4)(c), (7). ↩
-
AI Guidance (n 8). ↩
-
Privacy Act (n 1) s 6(1) (definition of ‘sensitive information’ paras (a)(vi)–(vii)), sch 1 cl 3.3; AI Guidance (n 8). ↩
-
Privacy Act (n 1) sch 1 cls 5.1–5.2; Court Data Australia (n 1) [49], [59]. ↩
-
Privacy Act (n 1) sch 1 cls 10.1–10.2, 11.1. ↩
-
Ibid sch 1 cls 1.3–1.4. ↩
Sources
- Privacy Act 1988 (Cth)
- Clearview AI Inc and Australian Information Commissioner [2023] AATA 1069
- Court Data Australia and Office of the Australian Information Commissioner [2025] ARTA 876
- Office of the Australian Information Commissioner, ‘Chapter 3: APP 3 Collection of Solicited Personal Information’ in Australian Privacy Principles Guidelines (Guidelines, 13 May 2026)
- Office of the Australian Information Commissioner, Guidance on Privacy and Developing and Training Generative AI Models (Guidance, 21 October 2024)
- Office of the Australian Information Commissioner, ‘Statement on Clearview AI’ (Media Release, 21 August 2024)