Idea for faster, better reviews of apps

[jpjarosz]

Gizmo,

Have you considered introducing additional social review features (other the commenting capability)? Particularly, I'm thinking of users being allowed to add their assessment of software applications presented in a category.

Generally speaking, I've noticed the following maturation process tends occur for a new sw category:

1. A new sw page appears which:
   • primarily categorizes the software (by adding links to the SW), and
   • tries to distinguish features of each app relative to others in the group (pairwise comparison).
2. Over time, the page's editor builds his own review content.
3. Feedback/comments are made by users, and the page's content gets incrementally updated by the page editor; new software is added to the review as users make the page editor aware of the left out sw.

A different concept/process could rapidly accelerate the maturation of the reviewed sw

1. A sw page editor is assigned and a special solicitation page is setup for a request of the apps existing in a particular category. A list of submissions are displayed on the page to reduce duplicate submissions of the same sw.
2. The editor evaluates the software made known, builds a comprehensive list of features and sets up a features matrix, rating each sw relative to its peers (pairwise comparison).
3. The editor writes an initial review to capture things that aren't be very well captured by a matrix and distinguishes certain features which stand out above the crowd.
4. The solicitation page is replaced by the review page, which consists of
   • a text area for the review of an application
   • a rating by features
   • a rating overall.
   
   Users could separately rate by their overall impression of the application and by feature (to provide the user the discretion of whether or not they want to participate in a detailed review, and also b/c mathematically adding up ratings of individual features for the overall score isn't valid due to each person prioritizing certain features differently; each person will also have a differing scale per feature, but this cannot be avoided). The page can display the editor's scoring separate from the users scoring to allow the reader to make decisions based on the opinion that suits them most (expert vs. consensus).
   
   Finally, the page also includes links to two different web forms:
   1. a form that allows users submit left-out/new apps for the category (form includes text boxes for sw name & URL)
   2. a form that allows users to submit a feature they think was not represented in the editor's matrix
   Submissions get routed to the editor to evaluate and add his initial input to the page. The editor still acts as the checkpoint in applying criteria as to whether or not something is a unique feature, or whether an application aligns properly to the SW category.

Eventually, the ability for users to add their own screencast/video reviews could also be added for each reviewed sw (though, I wouldn't suggest this as something to attempt initially).

So there ya go... my two cents on a few ways to make this review site even sweeter!

Good job guys and my highest regards to all! I love how this site has become better & better over time.

Jason

[MidnightCowboy]

Thanks for the input Jason, I'm sure Gizmo will be grateful for the suggestions.
Always, the main concern here is how best to manage a site staffed by 80+ volunteers. In many areas accepted processes often just aren't possible. This doesn't mean though that we are blind to change and always take our members and visitors suggestions seriously. Managing what we do have already is also a challenge when the team is spread out all over the planet and therefore in different time zones.

Ever given any thought to coming on board and helping us out?

[jpjarosz]

Originally Posted by MidnightCowboy

Thanks for the input Jason, I'm sure Gizmo will be grateful for the suggestions.
Always, the main concern here is how best to manage a site staffed by 80+ volunteers. In many areas accepted processes often just aren't possible. This doesn't mean though that we are blind to change and always take our members and visitors suggestions seriously. Managing what we do have already is also a challenge when the team is spread out all over the planet and therefore in different time zones.

Ever given any thought to coming on board and helping us out?

I have--I wish I had the time! A wife and two kids under the age of two take all my "free" time. Maybe in a few years.

[wildman]

How much vesting does Gizmo's do before it recommends a product? Who if anyone checks on the reputation of the companies supplying recommended products?

Thanks
Wildman

[kendall.a]

I will be the first to admit that when I review products/software, I am mainly concerned about it's functionality. I really do not delve into the software company or developer themselves. I see my job as evaluating different types or versions of software for the average or causal user. I do not see my job as evaluating the developer or company. Maybe that's wrong. I don't know.

To me, it would kind of like evaluating mutual funds. I would evaluate their performance and how well they are doing. It is up to the reader or investor at that point to determine if they want to do business with this company or decide if they don't agree with the companies mission. (For example, I know many people who choose not to invest in mutual funds with companies who support abortion, or support defense contractors, or don't support global warming, etc.)

This is just my opinion and how I see my role. My role is evaluating software and functionality, not the company themselves. There are people out there who might not want to support a company doing business in China or India or Korea or wherever for whatever reason. That is their choice and their issue. (For example, personally I will not support the Ford Motor Company by purchasing any of their vehicles. It has nothing to do with their value or even how good they are. However, that is my choice. It is not my job to push my values or beliefs onto others. However, if asked, I might or might not decide to give the reasons for my choice.)

[MidnightCowboy]

Originally Posted by wildman

How much vesting does Gizmo's do before it recommends a product? Who if anyone checks on the reputation of the companies supplying recommended products?

Thanks
Wildman

This is certainly a nice thought but I would be interested to know how we might achieve this considering our financial resources are nil. Even if we did possess such a budget, having some experience of the intelligence industry I've watched even government agencies throwing hundreds of thousands of dollars into validation procedures and still end up chasing ghost ID's and accommodation mailing addresses.

In short, we do try with what we have available but at the end of the day that's just us. We rely a lot on community feedback and our own experience to sift through what is presented. This process must work because very rarely do we ever have to retract anything, although we're always open to suggestions for improvement and offers of extra help

In reality, confirming truth is only easy if you deal with machines or animals.
As soon as you introduce humans into the equation then the whole process becomes a lot more complicated.

[wildman]

So let me see if I have this correct, All you are now doing is saying that this or that product does this or that function. You care less if the company behind the product is a known site for spam, adware, spyware or a harbinger for viruses. It's not your place to warm people of that possibility all you are here to do is let people know what the product does and you rely upon individuals to figure out the rest on their own. Do I have this correct?

Thanks
Wildman

[bk_7312]

Originally Posted by wildman

You care less if the company behind the product is a known site for spam, adware, spyware or a harbinger for viruses. It's not your place to warm people of that possibility all you are here to do is let people know what the product does and you rely upon individuals to figure out the rest on their own.

I seem to remember TSA said something about WOT ratings, if a site is known for spam, adware, spyware or a harbinger for viruses then it would very likely have a bad WOT rating meaning it wouldn't be listed here on this site.

I also remember that TSA had recommended their users to use WOT and stay away from sites with bad WOT ratings.

Yes, WOT isn't perfect but at least it helps.

[MidnightCowboy]

Originally Posted by wildman

So let me see if I have this correct, All you are now doing is saying that this or that product does this or that function. You care less if the company behind the product is a known site for spam, adware, spyware or a harbinger for viruses. It's not your place to warm people of that possibility all you are here to do is let people know what the product does and you rely upon individuals to figure out the rest on their own. Do I have this correct?

Thanks
Wildman

No you do not have this correct, and out of respect for all the volunteers here at TSA I would appreciate your not attempting to flame this thread by making inferences which are obviously not true. We take a lot of trouble to ensure as far as is possible within our means that what we present is reliable and from a safe source. This includes WOT rating websites and scanning download links too. I was merely suggesting that no matter how many resources you throw at this situation you can never guarantee 100% security. Even Kasperky was hacked a while back which I'm sure you know.

In reality half the universe seems to surf sites riddled with scams and malware such as Youtube, Facebook and other social networking sites and take little or no precautions to protect themselves. Here at TSA we have both automated and manual processes in place to ensure as safe an environment as possible. For obvious reasons I will not be revealing what these are.

[wildman]

Originally Posted by MidnightCowboy

No you do not have this correct, and out of respect for all the volunteers here at TSA I would appreciate your not attempting to flame this thread by making inferences which are obviously not true. We take a lot of trouble to ensure as far as is possible within our means that what we present is reliable and from a safe source. This includes WOT rating websites and scanning download links too. I was merely suggesting that no matter how many resources you throw at this situation you can never guarantee 100% security. Even Kasperky was hacked a while back which I'm sure you know.

In reality half the universe seems to surf sites riddled with scams and malware such as Youtube, Facebook and other social networking sites and take little or no precautions to protect themselves. Here at TSA we have both automated and manual processes in place to ensure as safe an environment as possible. For obvious reasons I will not be revealing what these are.

I will stop, I think the viewers of this forum have enough info. to now make up their own minds. I refer you to Kemdall's post #5 in this thread, I think it speaks for itself. I think your replies have also given enough info. Last word ethical or unethical site? The rest should be mute.

Thanks
Wildman