Please provide a short name/title for your Smart Data use case idea
Create a National Social Housing Database, combining Open Data with Technology Standards. To drive rapid improvement in the management of social housing, transforming the customer experience while reducing costs.
Smart Data is the collective term for technologies and policies that will enable consumers and small businesses to securely share the data that organisations hold on them with other organisations they trust, so that these can provide them with innovative services.
Please provide a short overview of your Smart Data use case idea
Ensuring the safety, quality, and efficiency of homes is of paramount importance in the social housing sector. However, most Housing Associations and Local Authority Housing departments face significant challenges due to a lack of reliable, real-time data. This deficiency not only hampers their operational capacity but also impedes their ability to effectively address resident needs and preferences. Moreover, it restricts their decision-making processes, inhibiting strategic changes, efficient resource allocation, and performance evaluations.
Recognising the urgency of this issue, both the Regulator of Social Housing and Ombudsman have acknowledged that the lack of dependable data undermines their regulatory effectiveness and, more importantly, creates confusion and delays in tenant-landlord interactions.
Compounding this challenge is the legacy data accumulated over six decades, entered through various platforms, including WhatsApp, texts, emails, and paper records. This fragmented data environment falls short of modern requirements and is managed using conventional tools such as spreadsheets, physical folders, email threads, and institutional memory. Consequently, many decisions are made without up-to-date information on issues such as building safety, damp and mould, thermal efficiency, etc.
Our Smart Data use case is to create a singular, focused database, standardising data structures, formats, and connectivity across local authorities and housing associations. This initiative would enable seamless data sharing between diverse platforms and organisations. We want to be an advocate for a collaborative approach in developing the necessary technology specifications, involving the Regulator of Social Housing (RSH) and government bodies, with support from the Office for National Statistics (ONS) and Government Digital Service (GDS).
Our vision is to make this standardised data framework mandatory for all stakeholders. Building upon draft common standards proposed by HACT, private sector companies will be able to rely on a clear specification and a guaranteed marketplace for their solutions.
Who is the end user (or users) that your Smart Data use case idea is intended for?
The end users and beneficiaries include:
a) Social housing tenants (who need comprehensive information about their home).
b) Government regulators (who can track, compare, and investigate property-related issues).
c) Social landlords (who can benchmark performance, collaborate, and make decisions based on reliable data).
d) Suppliers to the sector (who will have common standards they can depend on).
Currently, social landlords employ various methods for data collection, system storage, and visualisation. Many rely on spreadsheets, leading to insufficient data governance and integrity, resulting in errors and risks.
What is the end user’s need or problem that your Smart Data use case idea addresses?
Systems Don’t Communicate with Each Other:
Within Housing Associations (HAs) and Local Authorities (LAs), each existing system operates in isolation. Many of these systems are incompatible, preventing the seamless flow of data between different platforms and departments. Consequently, housing providers encounter difficulties in delivering smooth services to tenants, senior managers and boards are unable to make data-driven decisions, and regulatory compliance and audits unnecessarily become complex.
Skills, Capacity, and Capability Gap:
The social housing sector faces challenges in terms of skills and capacity when it comes to effectively utilising existing data resources. Each organisation is compelled to reinvent the wheel because there is no standardised, user-friendly system.
Outdated Data Management Tools, Data Fragmentation, and Duplication of Effort:
Housing data is dispersed across multiple platforms, including WhatsApp, texts, emails, and paper records. This fragmentation results in lost data, incomplete information, and compounding errors. The absence of a unified data solution forces staff to spend valuable time manually cross-referencing data from different sources. For instance, responding to government requests for specific data, such as identifying properties over 18 meters or those with damp and mould, becomes a time-consuming and error-prone task due to these outdated tools.
Can you quantify the benefits for the end user at all? Can you estimate how many end users your use case idea could be relevant.
The proposal would benefit up to 4.4 million tenants of social housing properties, over 300 local authorities, more than 1,500 housing associations, the Department for Levelling Up, Communities and Housing (DLUCH), the Regulator of Social Housing (RSH), and the Ombudsman. It would also have a positive impact on at least 250 small and medium-sized enterprises (SMEs) that supply the social housing sector.
In addition to the positive impacts on efficiency and cost savings, open data and transparency would significantly amplify the tenant’s voice.
In your opinion, could your use case contribute to any of the following government priorities, or have any other wider social benefits?
Cost of Living – Anything that reduces the cost of housing management and maintenance alleviates the pressure on social rent inflation.
Net Zero – A comprehensive database containing the Energy Performance Certificate (EPC) status of the entire social housing stock would encourage actions aimed at achieving net-zero emissions.
Levelling Up – Full visibility of the relative condition of social housing across the country would provide valuable evidence to identify areas in the greatest need of investment, potentially establishing an evidence base for reallocating resources. It’s worth noting that BAME individuals and people with disabilities are overrepresented in social housing.
Vulnerable Customers – Once a property database is established, it could be extended to provide quality (anonymised) data on vulnerable residents.
SMEs – This would create opportunities for smaller tech startups to access a vast data resource, enabling them to generate valuable insights for the sector and government.
Other – Regulation of the sector is risk-based. This use case would quantify the risk as it pertains to social housing dwellings in the UK, ensuring that regulation is more rational and targeted. With a standardised database in place, regulators can proactively take legitimate action against failing landlords before human tragedies occur.
Have you identified a potential commercial model for the use case?
Yes, we have identified a potential commercial model for repairs and maintenance contractors once we establish a sustainable and value-driven strategy. There will be significant potential for innovation and efficiency in delivering essential services to social housing.
Precise Maintenance Planning
Access to a comprehensive national database will enable repairs and maintenance contractors to enhance their planning and scheduling of maintenance activities. They can identify properties with specific needs, prioritise tasks based on urgency, and allocate resources efficiently.
Functional and Cluster-Based Insights
Data analytics can assist contractors in identifying patterns and trends in property conditions. By recognising common maintenance issues or vulnerabilities across clusters of properties, contractors can proactively address these issues before they escalate, reducing the need for reactive and costly emergency repairs.
Resource Allocation
Contractors can optimise their resource allocation by identifying geographical areas with higher maintenance demand. This allows them to establish more efficient routes and coverage, ultimately reducing travel time and costs. Like Uber’s algorithm that matches drivers with riders, contractors can use data to match available resources with maintenance requests.
Innovation Hubs
Clusters of similar properties can serve as innovation hubs where contractors collaborate with technology partners to develop and test innovative maintenance solutions.
What kinds of data would your Smart Data use case idea need in order to become operational?
Our use case relies on a diverse range of data sources, most of which are already in use within the day-to-day operations of social housing providers. Information about property locations is accessible through the National and Local Address Gazetteer, the Postcode database, and the Land Registry. We also intend to utilise the Unique Property Reference Number (UPRN) provided by Ordnance Survey to improve the accuracy of property identification. Additionally, we will incorporate the Energy Performance of Buildings Data for England and Wales, which is invaluable for assessing the environmental impact of properties and making informed decisions regarding energy-efficient upgrades.
Would you like to provide any further information about the data your Smart Data user case idea would need in order to become operational?
Golden Marzipan, a strategic technology consultancy specialising in social housing, leads this use case on behalf of a collaborative consortium that includes data experts, housing association stakeholders, potential suppliers, and other relevant agencies. This collective effort is driven by a shared commitment to revolutionise the social housing sector through the National Social Housing Database.
To facilitate rapid progress and comprehensive engagement, we have established a dedicated steering committee. We are in the process of setting up a Community Investment Company (CIC) to oversee the project. This committee/CIC plays a pivotal role in coordinating with key stakeholders, including housing associations, local authorities, regulatory bodies, technology partners, and potential end-users. Additionally, the steering group has initiated productive discussions with officials and representatives from the Department for Housing, Local Government and Communities (DHLGC).
We are pleased to report that our efforts have garnered significant support and commitment from these organisations. They have pledged their support by providing essential assistance and access to critical data. This collaborative synergy ensures that we can confidently advance in the development of a working model and Minimum Viable Product (MVP), including a foundational minimum dataset and adherence to standardised protocols.
We are actively identifying and consolidating all relevant data sources, conducting meticulous assessments of data quality, completeness, and compatibility. As we progress, we remain dedicated to transparency, collaboration, and excellence, with a shared vision of delivering transformative benefits to the social housing community and the broader ecosystem.
Who or what could help you refine or test your Smart Data use case idea?
The steering group has proactively anticipated engagement with interested organisations to ensure effectiveness and feasibility.
We have established contact with housing associations and local authorities that possess an understanding of the specific needs, challenges, and data requirements. These entities are open to the idea of collaboratively developing a more consistent asset management database, initially on a small scale.
Furthermore, we have initiated discussions with regulatory bodies such as the Regulator of Social Housing (RSH) and the Housing Ombudsman to solicit their input and gain insight into the regulatory landscape and any data reporting requirements. Multiple technology companies and software developers specialising in data management and analytics have expressed interest in participating in the project. Some of them have established relationships with regulators and can provide expertise in building the technical infrastructure necessary for the National Social Housing Database. Notably, one company has already conducted significant research and development related to existing open data in the UK property environment.
Is there any other information you would like to provide to us about your Smart Data use case idea?
Our objective is to harmonise, standardise, and integrate these diverse data sources to establish a unified and comprehensive National Social Housing Database. This initiative aims to ensure data accuracy, accessibility, and security while simplifying data management processes for social housing providers.
The initial scope of our data requirements concentrates on low-risk GDPR-compliant property data, focusing on outcomes rather than detailed processes. This data has the potential to replace and streamline some existing data collections conducted by regulatory bodies, such as the Regulator of Social Housing (RSH).
The key categories of data necessary to make our Smart Data use case operational include:
Property Ownership and Location:Â Information regarding property ownership, precise geographic location, and tenure types.
Property Characteristics and Age: Data describing the physical attributes and age of properties, which are critical for assessing maintenance needs.
Condition of Property: Data indicating the condition of properties, including whether they meet ‘decent homes’ standards.
Energy Efficiency and Heating Source:Â Information on the energy efficiency of properties and the source of heating, contributing to sustainability and energy-saving efforts.
Safety Risks:Â Data related to safety risks, encompassing gas, electricity, fire hazards, and issues such as dampness and mould.
In our research, we have identified potential sources of open data that could support, replace, or complement the data we require. Some of this data is collected in various formats by agencies collaborating with the Department for Housing, Local Government, and Communities (DHLGC), including the RSH Social Housing Data Return (SDR), data from the Housing Ombudsman’s Complaints, and data from Homes England.
Share your resources or write a blog about your data experiences in social housing.
We will publish your stories on our Insights page.
We also intend to curate and maintain a page of the most useful resources for data practitioners in the sector.
If this is successful, we may establish a subscription-based channel for buying and selling resources and services.
We use cookies to ensure that we give you the best experience on our website. If you continue to use this site we will assume that you are happy with it.OkPrivacy policy
