Create dynamic news site optimized for Google

Question

I want to create pages on a specific news site and I thought initially to store in a database the news information (title, subject, news text, etc). As soon as my page noticia.aspx was accessed, would search the content of the news in the database.

But this is not good practice for Google search engines to find my page. I also noticed that sites like Uol, Globo.com present the news page with the extension "HTML", as an example below:

http://g1.globo.com/economia/mercados/noticia/2015/09/cotaca-do-dolar-250915.html

How do these sites manage this news? Do they generate static pages? How to generate this content dynamically and visibly for Google?

Answer 1

Where did you get the information that this is not good practice? There is no problem in doing this. All sites do, the search engine accesses the page as if it were a browser.

How page generation works

I just don’t say "if the browser sees the content the search engine will also see" because this is not an absolute truth when the content is mounted through Javascript.

There is the myth that today this is solved, but it is not true. The situation that was zero ability of the mechanisms understand JS, improved, but still has ground. And yet it has its own techniques to achieve a reasonable result in some mechanisms, not all. Although only Google actually counts.

Any page generated on-the-fly on the server will be seen normally without any problem. The only reason to generate static pages would be to optimize the server capacity since you would generate a page once and then only make accesses. But I’ve seen people abuse this and get the opposite result.

When the page is dynamic it is generated every time it is requested, unless cache is used. And obviously the general page cache only works if the page never changes, or at least changes little, which is not so common. Often each page requested have a difference to the previous, by advertising, novelties, personalization or other reason. Dynamic pages generate more load on the server, but in most cases this is no problem, and the various levels of cache will help mitigate a very intense load.

Browser cache is also your friend.

The problem is on these pages SPA. In this case the generation is done in the browser, by JS. The search engine may simulate some executions and generate content, but there are several situations that this is impractical.

Extension of page name

When you generate a page you can put the extension you want. It is not required to be .aspx, or .php or another extension for it to be processed.

If you set up the HTTP server for an extension to be processed, every time something with this configured extension is called will trigger a designated processor, which can be an application or interpreter of a language. You can configure to the .html be processed as an ASP.NET application if you like. It has been agreed that the extension .html is used for static pages, but nothing prevents it from being this.

URL friendly

In addition it is possible to use a URL friendly (another question) which transforms the URL into a route that internally calls on the server some specific application or piece of it. That is, you disguise the actual address of the application for something that makes it easier for people to read, and search engines to identify the content. This is a very common technique that can be configured on the HTTP server (Apache, IIS, etc.) or within the application. Particularly prefer within the application.

Completion

Don’t worry about generating dynamic pages. And when this is creating load problems, start to see techniques to reduce the problem, but not before.

This is general information. When you have more specific questions open more specific questions.

Answer 2

The @Maniero response answers all the theoretical part behind it, I’ll just add a practical part of how to do it in Asp.net-mvc, since AP said that’s what he’s using.

To add the extension .html on their pages it is not necessary to create static pages, just use the route engine of Asp.NET MVC for that reason.

For example, if you’re looking for news, just do it in your controller:

[Route("{categoria}/noticia/{ano}/{mes}/{titulo}.html")]
public ActionResult Detalhes(string ano, string mes, string titulo)
{
    var noticia = db.Noticias.FirstOrDefatul(´/*where aqui*/);

    return View(noticia);
}

And in his file Web.config purchase this line:

<system.webServer>
        <handlers>
            <add name="HtmlFileHandler" path="*.html" verb="GET" type="System.Web.Handlers.TransferRequestHandler" preCondition="integratedMode,runtimeVersionv4.0" />
        </handlers>
    </system.webServer>

Done this, you will be generating URL's with the extent .html, but without static pages, as in the example quoted in the question.

http://g1.globo.com/economia/mercados/noticia/2015/09/cotaca-do-dolar-250915.html