[GH-ISSUE #607] [FEATURE] Add a parsable (JSON or XML) endpoint for results

kerem commented

2026-02-25 20:35:37 +03:00

Owner

Originally created by @throwaway34241 on GitHub (Jan 9, 2022).
Original GitHub issue: https://github.com/benbusby/whoogle-search/issues/607

Describe the feature you'd like to see added

Current the /search endpoint returns a HTML file to display the results. This works well for UI users, which might be the first use case of Whoogle. But it's hard for programmatic use cases, as it requires parsing HTML.

I propose to add a new /search.{json|xml} endpoint to return the same results in a structured format: JSON or XML

Idea 1: JSON

A /search.json?q=%s endpoint which would search results as JSON.

I don't think there's a standard JSON format to use, but we could take inspiration for https://zenserp.com/ or https://serpapi.com/.

Idea 2: OpenSearch XML format

This project already uses opensearch, so we could leverage the same standard's response XML: https://github.com/dewitt/opensearch/blob/master/opensearch-1-1-draft-6.md#opensearch-response-elements

Additional context

I have a personal preference for JSON, as it seems to be today's web standard for APIs.

Adding parsable results would allow integration with various projects, via API (e.g. Zapier, Integromat), RSS or simple scripts. Some use cases include:

keyword monitoring
market research
build a voice assistant
and the imagination of the community

Originally created by @throwaway34241 on GitHub (Jan 9, 2022). Original GitHub issue: https://github.com/benbusby/whoogle-search/issues/607  **Describe the feature you'd like to see added** Current the `/search` endpoint returns a HTML file to display the results. This works well for UI users, which might be the first use case of Whoogle. But it's hard for programmatic use cases, as it requires parsing HTML. I propose to add a new `/search.{json|xml}` endpoint to return the same results in a structured format: JSON or XML ### Idea 1: JSON A `/search.json?q=%s` endpoint which would search results as JSON. I don't think there's a standard JSON format to use, but we could take inspiration for https://zenserp.com/ or https://serpapi.com/. ### Idea 2: OpenSearch XML format This project already uses opensearch, so we could leverage the same standard's response XML: https://github.com/dewitt/opensearch/blob/master/opensearch-1-1-draft-6.md#opensearch-response-elements **Additional context** I have a personal preference for JSON, as it seems to be today's web standard for APIs. Adding parsable results would allow integration with various projects, via API (e.g. Zapier, Integromat), RSS or simple scripts. Some use cases include: - keyword monitoring - market research - build a voice assistant - and the imagination of the community

kerem

2026-02-25 20:35:37 +03:00

closed this issue
added the
enhancement

Fixed (Pending PR Merge)

good first issue
labels

kerem commented

2026-02-25 20:35:37 +03:00

Author

Owner

@vacom13 commented on GitHub (Feb 4, 2022):

@benbusby I will take it up

@vacom13 commented on GitHub (Feb 4, 2022): @benbusby I will take it up

kerem commented

2026-02-25 20:35:37 +03:00

Author

Owner

@sutidor commented on GitHub (Apr 7, 2022):

@vacom13 is there any beta to test?

@sutidor commented on GitHub (Apr 7, 2022): @vacom13 is there any beta to test?

kerem commented

2026-02-25 20:35:37 +03:00

Author

Owner

@vacom13 commented on GitHub (Apr 7, 2022):

@sutidor unfortunately there isn't yet 😢.

@vacom13 commented on GitHub (Apr 7, 2022): @sutidor unfortunately there isn't yet 😢.

kerem commented

2026-02-25 20:35:37 +03:00

Author

Owner

@benbusby commented on GitHub (Jul 13, 2022):

Closing in favor of #814, which includes an example JSON response that could be used as a model for Whoogle JSON results.

@benbusby commented on GitHub (Jul 13, 2022): Closing in favor of #814, which includes an example JSON response that could be used as a model for Whoogle JSON results.

kerem commented

2026-02-25 20:35:37 +03:00

Author

Owner

@throwaway34241 commented on GitHub (Dec 30, 2022):

#814 is a dead link for me, any new updated link?

@throwaway34241 commented on GitHub (Dec 30, 2022): #814 is a dead link for me, any new updated link?

kerem commented

2026-02-25 20:35:37 +03:00

Author

Owner

@benbusby commented on GitHub (Dec 30, 2022):

Huh, not sure what happened. I'll reopen this one.

@benbusby commented on GitHub (Dec 30, 2022): Huh, not sure what happened. I'll reopen this one.

kerem commented

2026-02-25 20:35:37 +03:00

Author

Owner

@Anduin2017 commented on GitHub (Mar 15, 2024):

Also expecting this feature :D

@Anduin2017 commented on GitHub (Mar 15, 2024): Also expecting this feature :D

kerem commented

2026-02-25 20:35:37 +03:00

Author

Owner

@zenoize commented on GitHub (Oct 10, 2024):

Also expecting this feature

@zenoize commented on GitHub (Oct 10, 2024): Also expecting this feature

kerem commented

2026-02-25 20:35:37 +03:00

Author

Owner

@emengweb commented on GitHub (Oct 25, 2024):

need json feature

@emengweb commented on GitHub (Oct 25, 2024): need json feature

kerem commented

2026-02-25 20:35:37 +03:00

Author

Owner

@Glossardi commented on GitHub (Jun 17, 2025):

I would love to use Whoogle but i need the JSON feature!

@Glossardi commented on GitHub (Jun 17, 2025): I would love to use Whoogle but i need the JSON feature!

kerem commented

2026-02-25 20:35:37 +03:00

Author

Owner

@VeryUsual commented on GitHub (Aug 23, 2025):

@benbusby #1237

@VeryUsual commented on GitHub (Aug 23, 2025): @benbusby #1237

kerem commented

2026-02-25 20:35:37 +03:00

Author

Owner

@Don-Swanson commented on GitHub (Sep 14, 2025):

Thanks @ProgramminCat
I'll review and test in the coming week. I'm trying to help Ben Catch up on PRs and issues, and starting with the easier ones.

@Don-Swanson commented on GitHub (Sep 14, 2025): Thanks @ProgramminCat I'll review and test in the coming week. I'm trying to help Ben Catch up on PRs and issues, and starting with the easier ones.

kerem commented

2026-02-25 20:35:37 +03:00

Author

Owner

@Don-Swanson commented on GitHub (Sep 24, 2025):

This has been incorporated into the Beta release: https://github.com/benbusby/whoogle-search/releases/tag/v1.0.0-beta

Please provide feedback if this is now resolved in beta.

@Don-Swanson commented on GitHub (Sep 24, 2025): This has been incorporated into the Beta release: https://github.com/benbusby/whoogle-search/releases/tag/v1.0.0-beta Please provide feedback if this is now resolved in beta.

kerem commented

2026-02-25 20:35:37 +03:00

Author

Owner

@JustAnotherRandomUsername commented on GitHub (Sep 28, 2025):

Please provide feedback if this is now resolved in beta.

Hello Don-Swanson

some bugs in json format?

screenshot

https://whoogle.pre.local/search?q=weather+colorado&format=json

{
  "query": "weather colorado",
  "results": [
    {
      "href": "https://support.google.com/websearch/answer/13687874",
      "text": "Wetterdaten"
    },
    {
      "href": "https://weather.com/weather/tenday/l/Denver+CO?canonicalCityId=678324881371c0fecd7a500615fc6991",
      "text": "10-DayWeatherForecast for Denver, COweather.com ›weather› tenday › Denver+CO"
    },
    {
      "href": "https://www.accuweather.com/en/us/denver/80203/weather-forecast/347810",
      "text": "Denver, COWeatherForecast - AccuWeatherwww.accuweather.com › denver › weather-forecast"
    },
    {
      "href": "https://www.accuweather.com/en/us/denver/80203/current-weather/347810",
      "text": "CurrentWeather"
    },
    {
      "href": "https://www.accuweather.com/en/us/denver/80203/weather-tomorrow/347810",
      "text": "WeatherTomorrow for Denver..."
    },
    {
      "href": "https://www.accuweather.com/en/us/denver/80203/daily-weather-forecast/347810",
      "text": "Daily"
    },
    {
      "href": "https://www.accuweather.com/en/us/denver/80203/hourly-weather-forecast/347810",
      "text": "Hourly"
    },
    {
      "href": "https://www.weather.gov/bou/",
      "text": "Denver/Boulder, CO - NationalWeatherServicewww.weather.gov › bou"
    },
    {
      "href": "https://www.9news.com/weather",
      "text": "Weatheron 9NEWS | Denver,Coloradowww.9news.com ›weather"
    },
    {
      "href": "https://weather.com/weather/tenday/l/Colorado+Springs+CO?canonicalCityId=c02dded5a48cf24a44c0eebaeac1d331",
      "text": "10-DayWeatherForecast forColoradoSprings, COweather.com ›weather› tenday ›Colorado+Spring..."
    },
    {
      "href": "https://www.cbsnews.com/colorado/weather/",
      "text": "Coloradoareaweatherand First AlertWeatherforecasts - CBS Newswww.cbsnews.com › CBSColorado›Weather"
    },
    {
      "href": "https://www.cbsnews.com/colorado/weather/colorado-futurecast/",
      "text": "ColoradoFutureCast"
    },
    {
      "href": "https://www.cbsnews.com/colorado/weather/current-alerts/",
      "text": "ColoradoAlerts"
    },
    {
      "href": "https://www.cbsnews.com/colorado/weather/severe-outlook-today/",
      "text": "ColoradoToday Severe..."
    },
    {
      "href": "https://www.cbsnews.com/colorado/weather/current-dew-point/",
      "text": "Dew Point"
    },
    {
      "href": "https://www.theweathernetwork.com/en/city/us/colorado/denver/current",
      "text": "Denver, CO CurrentWeatherwww.theweathernetwork.com › city ›colorado› denver › current"
    },
    {
      "href": "https://www.denver7.com/weather",
      "text": "Denver,ColoradoWeather| Denver7www.denver7.com ›weather"
    },
    {
      "href": "https://www.accuweather.com/en/us/co/colorado-weather",
      "text": "Local CurrentWeather- AccuWeatherwww.accuweather.com › colorado-weather"
    },
    {
      "href": "https://www.wunderground.com/forecast/us/co/colorado-springs",
      "text": "ColoradoSprings, CO 10-DayWeatherForecastwww.wunderground.com › forecast › colorado-spri..."
    },
    {
      "href": "https://www.google.com/intl/de_de/policies/privacy/",
      "text": "Datenschutzerklärung"
    },
    {
      "href": "https://www.google.com/intl/de_de/policies/terms/",
      "text": "Nutzungsbedingungen"
    }
  ],
  "search_type": ""
}

looks like json format have wrong numbers of results ...

HTML


<div class="ezO2md">
	<div>
		<div>
			<a class="fuLhoc ZWRArf" href="https://www.9news.com/weather" rel="nofollow noopener noreferrer">
				<span class="CVA68e qXLe6d fuLhoc ZWRArf">
					<b>Weather</b> on 9NEWS | Denver, 
					<b>Colorado</b>
				</span>
				<span class="qXLe6d dXDvrc">
					<span class="fYyStc">www.9news.com › 
						<b>weather</b>
					</span>
				</span>
			</a>
		</div>
		<div class="Dks9wf">
			<table class="KZhhub">
				<tbody>
					<tr>
						<td class="udTCfd">
							<div>
								<div>
									<span class="qXLe6d FrIlee">
										<span class="fYyStc">Hourly Temperatures · 1 PM. Mostly Cloudy. 75°. 4% · 2 PM. Partly Cloudy. 76°. 8% · 3 PM. Mostly Cloudy. 76°. 5% · 4 PM. Mostly Cloudy. 76°. 6% · 5 PM. Cloudy.</span>
									</span>
								</div>
							</div>
						</td>
					</tr>
				</tbody>
			</table>
		</div>
	</div>
</div>

It seems that the HTML is not being parsed correctly in app/routes.py lines 402-424

@JustAnotherRandomUsername commented on GitHub (Sep 28, 2025): > Please provide feedback if this is now resolved in beta. Hello [Don-Swanson](https://github.com/Don-Swanson) some bugs in json format? <details> <summary>screenshot</summary> <img width="757" height="637" alt="Image" src="https://github.com/user-attachments/assets/46495da3-1e18-4a6e-99ab-f8293341e5a7" /> </details> <details> <summary><code>https://whoogle.pre.local/search?q=weather+colorado&format=json</code></summary> ```json { "query": "weather colorado", "results": [ { "href": "https://support.google.com/websearch/answer/13687874", "text": "Wetterdaten" }, { "href": "https://weather.com/weather/tenday/l/Denver+CO?canonicalCityId=678324881371c0fecd7a500615fc6991", "text": "10-DayWeatherForecast for Denver, COweather.com ›weather› tenday › Denver+CO" }, { "href": "https://www.accuweather.com/en/us/denver/80203/weather-forecast/347810", "text": "Denver, COWeatherForecast - AccuWeatherwww.accuweather.com › denver › weather-forecast" }, { "href": "https://www.accuweather.com/en/us/denver/80203/current-weather/347810", "text": "CurrentWeather" }, { "href": "https://www.accuweather.com/en/us/denver/80203/weather-tomorrow/347810", "text": "WeatherTomorrow for Denver..." }, { "href": "https://www.accuweather.com/en/us/denver/80203/daily-weather-forecast/347810", "text": "Daily" }, { "href": "https://www.accuweather.com/en/us/denver/80203/hourly-weather-forecast/347810", "text": "Hourly" }, { "href": "https://www.weather.gov/bou/", "text": "Denver/Boulder, CO - NationalWeatherServicewww.weather.gov › bou" }, { "href": "https://www.9news.com/weather", "text": "Weatheron 9NEWS | Denver,Coloradowww.9news.com ›weather" }, { "href": "https://weather.com/weather/tenday/l/Colorado+Springs+CO?canonicalCityId=c02dded5a48cf24a44c0eebaeac1d331", "text": "10-DayWeatherForecast forColoradoSprings, COweather.com ›weather› tenday ›Colorado+Spring..." }, { "href": "https://www.cbsnews.com/colorado/weather/", "text": "Coloradoareaweatherand First AlertWeatherforecasts - CBS Newswww.cbsnews.com › CBSColorado›Weather" }, { "href": "https://www.cbsnews.com/colorado/weather/colorado-futurecast/", "text": "ColoradoFutureCast" }, { "href": "https://www.cbsnews.com/colorado/weather/current-alerts/", "text": "ColoradoAlerts" }, { "href": "https://www.cbsnews.com/colorado/weather/severe-outlook-today/", "text": "ColoradoToday Severe..." }, { "href": "https://www.cbsnews.com/colorado/weather/current-dew-point/", "text": "Dew Point" }, { "href": "https://www.theweathernetwork.com/en/city/us/colorado/denver/current", "text": "Denver, CO CurrentWeatherwww.theweathernetwork.com › city ›colorado› denver › current" }, { "href": "https://www.denver7.com/weather", "text": "Denver,ColoradoWeather| Denver7www.denver7.com ›weather" }, { "href": "https://www.accuweather.com/en/us/co/colorado-weather", "text": "Local CurrentWeather- AccuWeatherwww.accuweather.com › colorado-weather" }, { "href": "https://www.wunderground.com/forecast/us/co/colorado-springs", "text": "ColoradoSprings, CO 10-DayWeatherForecastwww.wunderground.com › forecast › colorado-spri..." }, { "href": "https://www.google.com/intl/de_de/policies/privacy/", "text": "Datenschutzerklärung" }, { "href": "https://www.google.com/intl/de_de/policies/terms/", "text": "Nutzungsbedingungen" } ], "search_type": "" } ``` </details> looks like json format have wrong numbers of results ... <details> <summary>HTML</summary> ```html <div class="ezO2md"> <div> <div> <a class="fuLhoc ZWRArf" href="https://www.9news.com/weather" rel="nofollow noopener noreferrer"> <span class="CVA68e qXLe6d fuLhoc ZWRArf"> <b>Weather</b> on 9NEWS | Denver, <b>Colorado</b> </span> <span class="qXLe6d dXDvrc"> <span class="fYyStc">www.9news.com › <b>weather</b> </span> </span> </a> </div> <div class="Dks9wf"> <table class="KZhhub"> <tbody> <tr> <td class="udTCfd"> <div> <div> <span class="qXLe6d FrIlee"> <span class="fYyStc">Hourly Temperatures · 1 PM. Mostly Cloudy. 75°. 4% · 2 PM. Partly Cloudy. 76°. 8% · 3 PM. Mostly Cloudy. 76°. 5% · 4 PM. Mostly Cloudy. 76°. 6% · 5 PM. Cloudy.</span> </span> </div> </div> </td> </tr> </tbody> </table> </div> </div> </div> ``` </details> It seems that the HTML is not being parsed correctly in <code>app/routes.py</code> lines 402-424

kerem commented

2026-02-25 20:35:37 +03:00

Author

Owner

@Don-Swanson commented on GitHub (Oct 1, 2025):

@JustAnotherRandomUsername
Thanks for catching that. I pushed a fix to the beta branch that you can test to verify. (I didn't publish a new release for this)

@Don-Swanson commented on GitHub (Oct 1, 2025): @JustAnotherRandomUsername Thanks for catching that. I pushed a fix to the beta branch that you can test to verify. (I didn't publish a new release for this)

kerem commented

2026-02-25 20:35:37 +03:00

Author

Owner

@JustAnotherRandomUsername commented on GitHub (Oct 1, 2025):

@JustAnotherRandomUsername Thanks for catching that. I pushed a fix to the beta branch that you can test to verify. (I didn't publish a new release for this)

Thank you! @Don-Swanson

    {
      "href": "https://www.9news.com/weather",
      "text": "Weather on 9NEWS | Denver, Colorado www.9news.com › weather Anonymous View Hourly Temperatures · 1 PM. Sunny. 76°. 0% · 2 PM. Mostly Sunny. 78°. 0% · 3 PM. Mostly Sunny. 79°. 0% · 4 PM. Partly Cloudy. 79°. 0% · 5 PM. Partly Cloudy. 78°."
    }

@JustAnotherRandomUsername commented on GitHub (Oct 1, 2025): > [@JustAnotherRandomUsername](https://github.com/JustAnotherRandomUsername) Thanks for catching that. I pushed a fix to the beta branch that you can test to verify. (I didn't publish a new release for this) <details> <summary>Thank you! @Don-Swanson</summary> ```json { "href": "https://www.9news.com/weather", "text": "Weather on 9NEWS | Denver, Colorado www.9news.com › weather Anonymous View Hourly Temperatures · 1 PM. Sunny. 76°. 0% · 2 PM. Mostly Sunny. 78°. 0% · 3 PM. Mostly Sunny. 79°. 0% · 4 PM. Partly Cloudy. 79°. 0% · 5 PM. Partly Cloudy. 78°." } ```

Rows
Columns

[GH-ISSUE #607] [FEATURE] Add a parsable (JSON or XML) endpoint for results #388

Idea 1: JSON

Idea 2: OpenSearch XML format