Judging Process
The judging process for the Australian Web Awards will be a two stage process based on a points system. The points awarded from the First Stage of judging are subtracted from the Second Stage of judging giving the final score.
This way the final score is reflective of validation, performance, accessibility and responsive design plus the peer judging. The quality of the sites that win will reflect the quality of the submission and the state of play in the industry. So it is possible for a site with invalid code to win. Unlikely, but still possible, it all depends what the other entries are like.
The First Stage
During the first stage all web sites will be evaluated for the following and awards on a points based process, All sites start with 0 points, as detailed below for each issue points are added. The sites with the lowest points (less issues) will progress to the next stage:
- Site Validation
- Site Performance
- Accessibility Compliance
- Responsive Design Compliance
We do not test the entire site, just several random pages.
So it is possible to remain on 0 points, or you entry may end up with 20-30 points, which isn't that good at all as you want a low score in stage one.
Site Validation
Validation is done to ensure that appropriate standards are adhered to. Points will be added as follows:
- For each unique ERROR – 3 points is added
- For each unique WARNING – 1 point is added
- For each unique BROKEN LINK – 1 point is added
Please note: that sites that all automated validation errors and warning will be manually checked as there are a number of EXCEPTIONS that we understand are out of the control of you as a developer.
We are judging your work, not your clients. Examples of such exception include:
- Sites controlled by Content Management Systems (CMS) controlled by the client; and
- Third party applications (such as JavaScript Widgets or Google Ads)
- Specification limitations based on current browser implementation short comings
If in doubt, please ask one of the Australian Web Awards Judging sub-committee for advice.
Site Performance
Having a site that performs well and renders quickly on any platform is critical for a good user experience.
Despite functionality all the web sites judged in the Australian Web Awards will be evaluated for performance via an automated process. Points will be added as follows:
- For site performance component that is below optium – 1 point is added. The performance components are rated 1-6, so it is possible to get 36 points here.
Performance will be measured as follows:
- Keep-Alive - Response headers with a "keep-alive" directive or were the same socket was used for more than one object from the given host, will get a pass score.
- GZIP Text - Transfer-encoding is checked to see if it is gzip. If it is not then the file is compressed and the percentage of compression is the result.
- Compress Images – Mainly dealing with Jpeg images – Any image within 10% of a photoshop quality 50 will pass. The overall score is the percentage of image bytes that can be saved by re-compressing the images.
- Cache Static – If an "Expires" header is present (and is not 0 or -1) or a "cache-control: max-age" directive is present and set for an hour or greater then a pass will be awarded.
- Combine CSS/JS - If multiple files of the same type are served then each additional css file beyond 1 will subtract 5 percent and each Javascript file beyond the first will subtract 10 percent.
- Use A CDN – Each page element will; be checked to see if it is hosted on a known CDN (CNAME mapped to a known CDN network). The known CDN's are Akamai, Amazon CloudFront, Coral Cache, Edgecast, Google, Highwinds, Internap, Limelight, Mirror Image, Panther and Yahoo.
Accessibility Compliance
Accessibility should be considered as part of an inclusive design process. To this end all sites must have a reasonable degree of accessibility compliance. Apps too should still make an attempt to comply to WCAG2.
The level of WCAG 1 or 2 compliance that you have selected in your application will be tested. Points will be added as follows:
- For each unique ERROR – 1 points is added removed
Responsive Design
It is assumed that all sites (not apps) entering in the web awards will render well on a vartitiy of platforms from the desktop to mobile computing devices. If a site does not render well with a purpose build or responsive design then the following points will be removed:
- For lack of purpose build or responsive design – 1 point is removed.
Elimination from the Awards
Any site that gets more than 50 points in stage one is automatically eliminated from the awards.
The Second Stage.
Progression to stage two of the judge depends on how high you scored in stage one. The lower the score the better.
However be very aware that progression to stage two is dependant on the number of entrants and their scores. If your entry scored highly it is unlikely it will progress in the rest of the pool of entries.
During the second stage of the judging process a panel of judges will evaluate the sites against the six general criteria detailed below.
- Visual Design
- Content
- User Experience
- Development
- Credibility
- Accessibility
Different categories have the criteria weighted differently accordingly. As such, not all criteria are relevant to all categories, and may be omitted from those categories weighting.
Again, if in doubt, speak to one of the Australian Web Awards Judging sub-committee members for advice.
Visual Design
- Is the website aesthetically pleasing?
- Does it create meaningful visual interest?
- Is it visually suitable for its purpose?
- Is the colour choice appropriate for the target market?
- Is the website designed to appeal to the target market?
- Does the visual design represent the image /persona of the business or agency?
- Is the branding of the business or agency consistent?
- Does the website use consistent elements and appropriate graphics and images?
- Typography:
- Does the website make use of ‘white space’?
- Are the size and style of fonts appropriate?
- General Layout of the page and alignment use visual design techniques to appear uncluttered.
- Does the website display consistently in various, widely-used web browsers.
- Does the website display in mobile or small screen devices?
Content
- Is the content written in clear and simple terms?
- Is the amount of content on each page appropriate? This means the content not being too wordy.
- Is the language and tone of the wording natural and familiar to the website’s target market?
- Is the content informative?
- Are keywords and key phrases used in a balanced way throughout the content?
- Is there limited use of unnecessary jargon and acronyms?
- Is the website free of spelling or grammar errors, broken links, out-of-date or obviously inaccurate content?
- Are appropriate policies included e.g. Customer Satisfaction Guarantee, Returns, Security, Privacy, Copyright easy to understand?
User Experience
- Is the content structured to help target market find information easily?
- Is the content mark-up appropriately? (This is not the code but headings, paragraphs etc)
- Are headings obvious and descriptive?
- Does the website have clear, predictable navigation with uncomplicated structure?
- Are internal links managed well? Is there a site map or search facilities where appropriate?
- Are the menu groupings logical and easy to follow?
- Does the website suit its target market and deliver its message in an engaging and appropriate fashion?
- Is it relevant, fulfilling / achieving its purpose / objectives?
- How do you measure success?
- Does it meet identifiable goals?
- Selling experience
- Is the shopping catalogue easy to browse or search?
- Are you able to register for product updates?
- Shopping Cart Flow (if applicable)
- Is it easy to add another product to the cart?
- How easy is it to create an account for the website?
- Is the checkout process uncomplicated and easy to use?
- How difficult will it be for the target audience to use the site
- Does the website technologies function as expected?
- Are links to and from external sources managed well?
Development
- Does the website use any technologies other than HTML & CSS in the site. Technologies include databases, shopping carts, forms, payment processes, calculators, search processes, news and mailing lists.
- If JavaScript is implemented is it following best practice.
- Do the website technologies achieve the functionality requirements?
- Are the technologies simple, logical and easy to use?
- Do the various parts of the website download in an adequate time frame?
- Do the Title and Description meta-tags have adequate keywords?
- Is the website optimised appropriately for search engines?
- How innovative is the system or technologies?
- Does the system or technologies use technique/s in new and exciting ways?
- Will the system or technologies have a big impact on the target audience
- How unique is the system or technologies? This can be based on the uniqueness of the idea, User Interface or implementation
- Online Security provided for websites providing user registration/login facilities.
- Customisation
- is the site a built from the ground up implementation?
- is the site an out of the box implementation?
- is the site a customisation of an existing online store environment?
Credibility
- Does the website include a Privacy Policy (where required by the Privacy Act 1988 (Cth)
- Can customers access their information submitted to the website?
- Does the website include a Customer Service Policy or information?
- For Online Businesses are ethical issues such as goods return or refunds addressed?
- Is the website free of inappropriate content?
- Are contact details provided for customer enquiries?
- Does the website include contact names, addresses and phone numbers for the business?
- Is security of customers personal and financial information assured?
- Security Certificate type – What is the level of certificate installed on the site (if applicable)
Accessibility
- Does the website adhere to WCAG Priority One and Priority Two?
- Is the markup valid and semantic?
- Is there a separation of content, presentation and behaviour layering, making use of Cascading Style Sheets, and unobtrusive scripting?
- Is the website viewable in different browsers and different platforms?
Judges Comments
Again this year judges will be giving brief comments on each site.
These comments are aimed at improving and fostering the use of best practices within the web industry. The comments will remain anonymous, and NO correspondence will be entered into over these comments. This is being done as a service to the industry. We reverse the right to withhold any comments at any time.
These comments will not be sent out until after all the finalists have been awarded.
Supplimentary Hints and Tips
You are bound to have questions and issues with your entry, that's okay there is a good chance it's not something new. We suggest you check out:
Conflicts of Interest
We make every effort to remove any bias from the judging process. We go out of out way to have each judge declare publically conflicts of interest.
This year you may have noticed we have distanced ourselves (the Awards Committee) even further away from the Australia Web Industry Association (AWIA). During judging there will be no common members on the AWIA or the Awards Committee.
Support from Industry
This year professional organisations such as the Australian Web Industry Assocation (AWIA) and Web Industry Professionals Assocation (WIPA) are supporting the Australian Web Awards ensuring that these are afterall the Australian Web Awards for all of the professionals associated with the Web Industry.