|Written By: Jim Antler
|Originally Published in the March 1993
issue of The Outfitter Magazine.
In November 1992, NOTO sent out a survey to its members asking them for feedback on the new Canada Select Grading Program. The survey results have been compiled and are summarized below for the benefit of the entire membership.
Of the 425 surveys mailed to the membership, 56 were returned completed to the NOTO office. This translates into a 13% response rate, not too bad for a mail-in survey.
Generally, most respondents were unhappy with the new program and expressed reservations over being graded in 1993. However, the Quality of Experience check-mark rating system did garner some positive comments as a number of respondents felt it had merit and seemed to represent an improvement over the old Tourism Ontario Grading Program.
The survey results will be described in two parts. The first part will provide a general overview of the answers to the most basic survey questions (Questions 1-5 and 7). These questions were designed to obtain information such as:
How many properties were graded in 1991 and 1992;
Under what category were they graded in 1992;
The level of satisfaction with 1992 grades;
Whether or not properties were planning to be graded in 1993.
The second part will contain more detailed results as the responses to the survey questions will be broken down based on how the respondents answered the first two questions of the survey. These two questions asked if properties were graded in 1991 and 1992 respectively.
It is felt that by grouping the results together in this manner, it will give a better idea of how specific groupings of graded and non-graded properties felt about the program. Feedback is provided by respondents as they relate to the Quality of Experience ratings (Question 6) and any other comments they wished to make about the Canada Select Grading Program (Question 8) will also be summarized.
There are five response groups to be addressed under part two of the survey analysis. They are as follows:
Properties graded in both 1991 and 1992;
Properties not graded in either 1991 or 1992;
Properties graded in 1991 but not in 1992;
Properties not graded in 1991 but graded in 1992;
Properties that gave incomplete responses.
PART 1 - GENERAL OVERVIEW OF MOST ASIC SURVEY RESULTS
As noted earlier, 56 NOTO members responded to the survey. They responded to the following basic questions.
QUESTION ## 1: Were you graded in 1991 under the old Tourism Ontario Grading Program?
41 of the 56 respondents said YES (78%), 13 said NO (23%) and 2 did not answer the question (4%).
QUESTION ## 2: Were you graded under the Canada Select Grading Program in 1992?
35 of the 56 respondents said YES (62.5%), while 21 answered NO (37.5%).
It is also interesting to note that a total of 104 NOTO members were graded in 1992, based on information provided to NOTO by Tourism Ontario. Therefore, approximately 1/3 of all graded NOTO properties responded to the survey.
FOLLOW-UP TO QUESTIONS 1 & 2
The combination of responses for questions 1 & 2 were as follows:
32 properties were graded in both 1991 and 1992;
11 were not graded in either 1991 or 1992;
9 were graded in 1991, but not in 1992;
2 were not graded in 1991, but were graded in 1992;
2 gave incomplete responses.
QUESTION ## 3: Under what category were you graded in 1992?
Of the 35 respondents who were graded in 1992, 23 were graded under the HUNTING/FISHING category, 9 were graded under the RESORTS category and 3 were graded under the HOTELS/MOTELS category.
QUESTION ## 4: Which of the following best describes your star grade in 1992?
Of the 35 respondents, 28 received LOWER star grades in 1992 than in 1991, 3 received the same star grade, and 2 received a HIGHER star grade than in 1991. 2 respondents noted they were NOT GRADED in 1991.
QUESTION ## 5: How would you rate your level of satisfaction with the star grade you received in 1992?
Of the 35 respondents, 24 were NOT SATISFIED with their 1992 star grade, 10 were SATISFIED with their star grade, and 1 indicated NO OPINION.
QUESTION ## 7: Is it your intention to be graded under the Canada Select program in 1993?
Of the 56 total respondents to the survey, 26 said NO, 11 said YES and 17 were UNDECIDED. 1 respondent did not answer the question and 1 double marked the survey.
In summary, there was an overall dissatisfaction with the star grades received in 1992 under the new Canada Select program, as the vast majority of graded properties saw their star grades decline in 1992.
This dissatisfaction is also demonstrated through the apparent unwillingness of properties to be graded in 1993. The presence of a large undecided vote seems to indicate that a fair number of camps may be looking for more feedback on the program before committing themselves to be graded in 1993.
PART 2 – DETAILED BREAKDOWN OF SURVEY RESULT BASED ON ANSWERS TO QUESTIONS 1 AND 2
As indicated earlier, there were 5 different respondent groups based on how questions 1 and 2 were answered. By analyzing the specific responses to the survey questions relevant to each group, it will help determine if those properties who were involved in the Canada Select Grading Program have different views than those properties who were not.
GROUP A - Properties That Were Graded in Both 1991 and 1992
32 of the 56 total respondents fell into this category (57%).
22 were graded under the FISHING/HUNTING category, 7 under the RESORTS category and 3 under the HOTELS/MOTELS category.
28 of the 32 properties received a LOWER star grade in 1992 than in 1991. 3 properties received the SAME star grade and 1 received a HIGHER star grade in 1992.
22 respondents were NOT SATISFIED with their star grade in 1992. 9 respondents were SATISFIED with their 1992 star grade. 1 respondent indicated NO OPINION.
When these respondents were asked how they felt about the “Quality of Experience” (QOE) check-mark rating system under the new Canada Select program, their comments were varied. Not every property provided comments, but for those who did, their comments fell under the following broad categories.
10 respondents provided positive comments about the QOE rating system. Actual comments ranged from “satisfied” to “a valuable addition to the grading program” to “excellent…level of checks indicate for the first time the level of service we deliver”.
7 respondents provided negative comments about the QOE rating system and how the ratings are determined. Sample comments include:
“don’t like it - limited time spent with customers should be spent listening and building relationships - not to explain what you need from them”;
“confusing to potential guests”;
“this system is stupid - degrading to have to ask guests to fill them out”.
3 respondents did not receive a QOE rating because either they received ballots too late in season or their guests did not complete a sufficient number of ballots. 4 respondents took a “wait and see” approach, indicating they were not sure how effective the ratings may be, or that they would only be effective if the program was well known, understood by guests and marketed properly to the travelling public.
Other comments on the QOE system included: why are there not ½ checks if there are ½ stars; and, that quality ratings should be shown in stars, not check marks.
When asked if they planned to be graded again in 1993, 12 said NO, 9 said YES and 10 were UNDECIDED. One respondent double marked the survey.
Question ## 8 in the survey asked respondents to provide any other comments they wished to make on the Canada Select program. Extensive comments were received, including copies of many letters sent by specific properties to Tourism Ontario with questions about the specific star and check-mark ratings they received.
Overwhelmingly, the comments were negative in tone and dealt with all aspects of the program, including:
How it is administered;
The amount of information (or lack of information) provided to operators on the details of the program;
Problems with star and check-mark grades received;
Questions over the appropriateness of the criteria that operations are graded under (especially as they relate to housekeeping operations and fishing camps/lodges).
Some respondents indicated clearly that they would not participate in the program in 1993, with one property even asking for their money back from 1992.
There were some positive comments, however. One property felt that the Canada Select program should be an “even stronger” marketing tool than the old Tourism Ontario program because of the QOE ratings.
GROUP B - Properties That Were Not Graded in 1991 or 1992
11 of the 56 total survey respondents fell into this category.
Question ## 2 in the survey asked if properties were not graded in 1992, what was the reason for not being graded?
The reasons that these particular respondents provided includes:
They could not get enough information on the new program (i.e. criteria, cost, number of categories);
It was too expensive and perceived as not providing enough value for cost;
Potential clients had not asked them in the past whether they were graded;
Other grading programs already exist (i.e. AAA/CAA, Woodalls Campground ratings) so the new program was not needed.
2 of the 11 respondents provided comments on the QOE rating system. Both felt there was some potential in such ratings, but only time would tell.
When asked if they intended to be graded in 1993, 7 said NO, 3 were UNDECIDED and 1 said YES.
When asked to provide any other comments about the Canada Select program, some of the responses given under question ## 2 were repeated (i.e. too expensive, never been asked before if graded). One respondent also expressed a personal observation that camps with equal star grades seemed often to be quite varied in their quality.
GROUP C - Properties Graded in 1991, But Not Graded in 1992
9 properties fell into this category. When asked why they did not have themselves graded in 1992, 3 respondents indicated they just did not want to participate. Others gave specific reasons:
Do not agree with the new system;
Are problems with the grading criteria;
Concerned over the potential for a lower star grade.
Short of money to join.
When asked about the QOE rating system, 7 of the 9 properties gave no opinion. 2 felt they needed more information and time to see if they will be useful.
When asked if they planned to be graded in 1993, 6 said NO, 0 said YES, 2 were UNDECIDED and 1 did not answer.
Other comments provided on the Canada Select program included:
Problems with grading criteria/expectations which are too high for smaller properties;
Housekeeping properties are not categorized properly;
It will take a long time to renew industry confidence in the new grading program.
One respondent said that the new rating system seemed “excellent”, but has a concern that properties with a 3 star grade would not be considered “above average” by guests, but simply “average”, since 3 stars is mid-way between 1 star and 5 stars.
GROUP D - Properties Not Graded in 1991, But Graded in 1992
Both properties in this category were RESORTS who were NOT SATISFIED with their 1992 star grades. Both were also UNDECIDED as to whether they would be graded in 1993.
With regard to the QOE rating system, one respondent felt the system “has its merits”, while the other was unhappy that their ballots were not received until after Labour Day, with the result being that there were not enough ballots to get a QOE rating.
Again, concerns over perceived inappropriate grading criteria were identified when asked for other comments on the program.
GROUP E - Properties Giving Incomplete Responses
2 properties fell into this category. One did not indicate if it was graded in 1991, but did indicate it was not graded in 1992. This property did not plan to be graded in 1993. It also felt properties should not pay to be graded because it looks like grades are being “bought”.
The other property did not indicate if it was graded in 1991, but was graded in 1992. It was graded under the FISHING/HUNTING category, was SATISFIED with their 1992 star grade, and intends to be graded again in 1993.
This property also felt the QOE rating system provided “good input” into the level of guest satisfaction, however, it expressed concern that it was tough to get their guests to fill out customer service ballots because many guests did not want to be bothered.
SUMMARY COMMENTS ON SURVEY RESULTS
Overall, there was more negative response to the new Canada Select Grading Program than positive response. As noted above, the concerns about the new program were many and varied, dealing with almost every aspect of the program from administration to criteria to actual grades received.
There also seems to be a lot of reservation among respondents over whether to be graded in 1993. While the majority of decided properties do not want to be graded, there remains a significant number of properties who are as of yet undecided. However, based on the amount of negative comments that were submitted on the program, one may speculate that many of those undecided properties will not choose to be graded in 1993, unless they become more confident in Canada Select.
It would seem that the aspect of the program that received the most positive feedback was the Quality of Experience rating system. Many properties felt it had the potential to be very worthwhile and an improvement over the old Tourism Ontario program.
However, many properties were very disappointed over receiving their ballots too late and then not being able to get enough completed ballots to obtain a rating. Their disappointment was magnified because they were not informed that a minimum number of completed ballots were needed to get a rating, or what number was.
It also seems that both graded and ungraded properties have a number of similar opinions on the Canada Select program. This may indicate that there is a fair amount of discussion in the field about the program. However, as the responses show, whatever discussions that are occurring are primarily negative in scope.
After reviewing the survey responses and comments received, one gets a strong feeling that many properties felt the Canada Select program was implemented too quickly. Many respondents had concerns over a lack of information made available to them about the program. One also gets the sense that many properties in our sector of the industry felt the program administrators were somewhat unorganized with regard to the new system, leading to delays in receiving grades, delays in receiving ballots and ballot boxes and so on.
It would seem that the administrators of the Canada Select grading Program have a tremendous marketing and public relations task ahead of them to “sell” the program to our sector of the tourism industry. Many properties are clearly against the program and have no intention of being graded in 1993. Given the existing low number of graded properties in 1992, this can only worsen the plight of the grading program in 1993.
If the Canada Select grading Program is to survive in the long term, it would seem that a tremendous amount of work will have to be done to the program before industry confidence is regained.