Launching a web-based news aggregator

01 Nov 2018

Launch

We are two friends and we are happy and excited to launch Newshound, a web-based news aggregator. This is something that we have been working on for the past one year. We have released this publicly today in the hope that others would also find it useful. Ultimately, we hope that you make it a part of your daily lives, just like it is for us.

Main features

Newshound gathers news from multiple organizations that publish stories of national interest representing a broad spectrum of political views. An algorithm clusters related news stories and categorizes them into broad sections like politics, business, technology, sports, etc. Op-eds are in a section of their own, separate from straight news reporting. A separate algorithm identifies the top newsmakers of the day.

Roughly once an hour, the top headlines of the moment are published on the website newshound.co. The page loads quickly in the browser; it is about 1 MB total in size. It consists of a few hundred KB of pure HTML and CSS with no JS. The bulk of the page size consists of thumbnails that accompany the news stories.

Everyone gets to see the same news stories. All links go directly to the news publisher's website with no AMP and no redirection involved. There is no algorithm to track what you read and show you more of the same, so there is no filter bubble. The website does not use cookies. There are no ads. AWStats, an open source software for web analytics, is installed on the server side to collect anonymized, aggregated data about website visits. This data allows us to estimate usage and provision our web servers to handle the expected load.

How we got here

About a year ago, we started talking about creating a news aggregator that would show multiple sides to the same story with no tracking and no ads. We scratched our itch and soon had a prototype going. We shared it with friends and family and got a positive response. Both our dads have MBAs, so they encouraged us to “make it real”. One of our family friends who is an attorney helped us form an LLC.

When we started out building Newshound, we didn't know anything about the technology that powers news aggregators in general. The learning curve was steep but along the way we picked up quite a bit about text mining via natural language processing methods. We hope to share this learning with you in a series of engineering blog posts on our website. We're not experts by any means, so tips on how we can do better are most welcome!

Where we go from here

While we have overcome some technical challenges to launch our product, we have no business model that will help sustain it over a period of time. We have no desire to introduce ads on our website. We are not interested in tracking individual visitors to our website and selling that data to marketers. So we need to have regular users of our website and provide them features that they would be willing to pay for. We have some idea of what we can do here but we would love to have your feedback as well.

Here is a list of features that we think we can provide in the future, in no particular order:

  • Multiple news editions: We've launched a news aggregator with just one edition (for the U.S.) but we could do more, for example Canada, UK, Europe, Africa, India and Japan.
  • Local news and weather: We could enable users to create a profile and select a city for which they would like to see the local news and weather conditions.
  • Search news archives: We could code up a search interface to our database of news stories, enabling users to see how a particular news topic evolved over time.
  • Sentiment analysis: We could attach a sentiment (positive, negative or neutral) with each story, allowing users to track a particular topic and get a feel of how it is being talked about in the press.
  • Customized news: We could enable users to add their own news sources, remove existing news sources, and rearrange news sections to achieve a custom news page for their daily browsing.
  • News API: We could build an API for our database of news stories, enabling developers to come up with some really cool products.
  • Stockmarkets: For those invested in the stock market, news that moves the markets or the share price of their stocks is very important. We could hook up with a provider of stock market data and enable our users to track their portfolio over time.

Get Involved

If you like our work and feel that it has legs, please help us to keep it going. You can get involved in a number of ways big and small depending on your inclination.

  1. If you are interested in U.S. news, please bookmark our U.S. edition news page and make it a part of your daily browsing. Statistics on daily website visitors will help us track traffic growth and provision our servers and bandwidth accordingly.
  2. If you are interested in receiving the day's top headlines in your email inbox, please sign up for our daily email. A growing number of email sign-ups will tell us that we are providing a valuable service and encourage us to keep going.
  3. If you are interested in news editions other than the U.S., please tell us which others you would like to see. Please participate in this poll and vote for your favorite option or suggest a new option.
  4. If you are interested in news sections other than the ones we already provide, please provide your suggestions in the comments area below along with news sources for them.
  5. If you feel that we need to add more news sources to add diversity to our news coverage, please provide your suggestions in the comments area below.
  6. If you want to hear more about the technical challenges that we faced while developing the news aggregator, please subscribe to our Engineering blog via RSS. We'd love to take you on a tour behind the scenes and pick up tips from you on how to do better.
  7. If you feel that can help guide us towards a self-sustaining business model that does not rely on advertising and selling users' personal data, please put our future roadmap under the lens and tell us how we can build features that you will be happy to pay for. Please participate in this poll and vote for your favorite feature or suggest a new feature.
  8. If you would like to see us keep going while we explore a self-sustaining business model, please donate to keep our lights on and servers running.

Finally, a big thank you for making it all the way to this point. Your time is greatly appreciated. We hope you enjoy reading Newshound as much as we have enjoyed putting it together!